@researai/deepscientist 1.5.16 → 1.6.0

package/src/skills/write/references/experiments_analysis_patterns.md

@@ -0,0 +1,129 @@
+# Experiments And Analysis Patterns
+
+Use this reference when strengthening the `experiments` and `analysis` sections of an oral-style ML paper.
+
+The main objective is not to report more numbers. It is to make the empirical package reviewer-auditable, interpretable, and memorable.
+
+## 1. Separate empirical proof from interpretation
+
+When the staged package is rich enough, keep `experiments` and `analysis` as distinct jobs.
+
+- `Experiments` proves where the method works, against whom, and under what settings.
+- `Analysis` explains why the pattern is credible, what tradeoff it reveals, and where the claim boundary lies.
+
+Do not let `analysis` become a short epilogue after the main result dump.
+
+## 2. Experiments should answer explicit reviewer questions
+
+A strong experiments section usually reads as a sequence of local questions rather than a flat list of tables.
+
+Typical reviewer-facing questions:
+
+- Does the method beat named baselines on the main task?
+- Does the gain persist across datasets, scales, or settings?
+- Does the gain survive stronger baselines, retuning, or ablations?
+- What practical cost, latency, or data tradeoff accompanies the gain?
+
+Each question should have one dominant display or authored compact table. Do not force one omnibus table to answer all of them.
+
+## 3. Preserve an inspectable main-text comparison surface
+
+Reviewers should be able to audit the main claim from the main text.
+
+Keep in the main paper:
+
+- the principal benchmark comparison
+- the main baseline families or evaluation regimes
+- at least one non-headline support block when the central claim needs visible transfer, robustness, or objection-handling evidence
+
+Avoid these failure modes:
+
+- prose claims that mention wins without a visible comparison surface
+- averaged or compressed tables that hide which baselines matter
+- moving all boundary-setting evidence into appendix overflow
+
+## 4. Give different displays different jobs
+
+When several staged artifacts are available, assign complementary roles instead of redundant summary roles.
+
+Useful experiments display jobs:
+
+- headline benchmark block
+- robustness or transfer block
+- ablation or intervention block
+- practical-cost or efficiency block
+- human-evaluation or qualitative support block
+
+Useful analysis display jobs:
+
+- mechanism or credibility check
+- tradeoff or sensitivity check
+- failure-boundary or scope-limit check
+- category breakdown or taxonomy check
+
+If two figures or tables support different reviewer concerns, preserve that distinction in the manuscript.
+
+## 5. Analysis needs a visible main-text evidence floor
+
+The main text should not outsource the whole analysis layer to appendix citations.
+
+When the staged package supports it, keep at least:
+
+- one mechanism or credibility display
+- one tradeoff, sensitivity, robustness, or quality-support display
+
+These can be compact, but they should remain inspectable in the main paper when they answer different reviewer concerns.
+
+## 6. Analysis should name the stable pattern
+
+Good analysis prose does more than narrate a chart.
+
+It should answer:
+
+- what trend remains stable across settings
+- what tradeoff is being managed
+- what mechanism most plausibly explains the pattern
+- what scope limit or failure boundary the evidence implies
+
+Weak analysis often cites correct numbers but never states the governing pattern.
+
+## 7. Use labeled analysis blocks
+
+When the gold or staged package has multiple interpretive jobs, make them visible through subsection heads, bold mini-heads, or strongly signposted blocks.
+
+Common analysis block types:
+
+- metric or proxy validation
+- mechanism explanation
+- diversity-versus-accuracy or quality-versus-quantity tradeoff
+- robustness across scales or model families
+- failure taxonomy or error-boundary analysis
+
+Do not compress these jobs into one generic "Further analysis" paragraph.
+
+## 8. Keep appendix bridges concrete
+
+Main text should hand off overflow support deliberately.
+
+Good bridge sentences name what lives in the appendix:
+
+- extra benchmark slices
+- protocol details
+- broader sweeps
+- qualitative examples
+- human-evaluation annotation details
+- additional failure cases
+
+Bad bridge sentences just say "see appendix for more results."
+
+## 9. Fast audit for experiments and analysis
+
+Before finalizing these sections, check:
+
+- Can a reviewer inspect the main comparative claim directly in main text?
+- Is there a separate non-headline block for robustness, transfer, ablation, or objection handling when the claim needs it?
+- Does analysis contain more than one visible reviewer-facing check?
+- Does analysis name the stable trend or mechanism instead of re-reading numbers?
+- Does the main text still carry visible analysis evidence, or did it drift almost entirely into appendix referrals?
+
+If any answer is no, the section package is probably too thin for oral quality.

package/src/skills/write/references/oral_package_patterns.md

@@ -0,0 +1,252 @@
+# Oral Package Patterns
+
+Use this reference to translate a dense draft into an oral-quality paper package.
+
+The core lesson is simple: oral papers do not just contain stronger claims. They stage evidence in a way that reduces reviewer friction and makes the contribution easy to retell.
+
+## 1. Complete persuasion chain
+
+A compressed draft often jumps from problem statement to winning numbers.
+
+An oral-ready paper usually gives the reader this sequence:
+
+1. context and why the problem matters
+2. deficiency in current methods
+3. the paper's core idea in plain language
+4. an early intuition or mechanism figure
+5. the main empirical result
+6. targeted analysis explaining why the result is credible
+7. appendix material that answers deeper verification questions
+
+If the draft skips multiple steps, the issue is structural, not stylistic.
+
+## 2. Evidence budget is part of the argument
+
+Reviewer trust depends not only on what evidence exists, but on whether the paper looks prepared for obvious follow-up questions.
+
+Strong oral packages usually spend visible budget on:
+
+- a clear main-text result section
+- at least one interpretive analysis section
+- a practical-value or objection-handling block when the main claim invites that question
+- appendix material for full tables, protocols, sweeps, proofs, and extended examples
+
+Weak packages often look too frugal:
+
+- too few figures doing too many jobs
+- one page carrying several unrelated claims
+- appendix used only for leftover tables
+
+Another common failure is misallocation rather than underspending:
+
+- too much main-text budget on repeated method defense
+- too little main-text budget on the objections reviewers are most likely to raise
+
+## 3. Reader onboarding before sharp claims
+
+Dense drafts often assume the reader already accepts the framing.
+
+Oral papers usually do more onboarding:
+
+- define the problem class before the exact gap
+- explain why the current family of methods is unsatisfying
+- give the idea in plain language before diving into formalism
+- use an early figure to make the later math easier to absorb
+
+This is not padding. It is trust-building.
+
+## 4. Display program: each important figure has a job
+
+High-value figure roles:
+
+- `Intuition figure`: appears early and explains why the method might work
+- `Main result figure`: establishes the central empirical pattern
+- `Interpretive figure`: explains a tradeoff, robustness pattern, or mechanism
+- `Case-study figure`: gives a memorable example, failure mode, or qualitative contrast
+
+Weak drafts often use figures only as result summaries. Oral papers use figures as teaching tools and memory anchors.
+
+When the paper is written section by section, decide this display program in the plan before drafting body prose. Do not let later sections discover displays opportunistically.
+
+## 5. Table program: one table, one reviewer question
+
+Tables should map to local claims rather than aggregate everything into one overloaded display.
+
+Good table roles:
+
+- main benchmark summary
+- human evaluation summary
+- efficiency summary
+- diversity or robustness summary
+
+If one table forces the reader to accept several unrelated conclusions at once, split it.
+
+## 6. Displays carry specifics, prose carries explanation
+
+One of the clearest differences between a draft and an oral-ready paper is the division of labor between figures or tables and the surrounding text.
+
+Use the display to carry:
+
+- exact numbers
+- full benchmark comparisons
+- local traces
+- concrete generations or examples
+
+Use the prose to carry:
+
+- the question the display answers
+- the dominant pattern the reader should retain
+- why that pattern appears
+- what scientific or reviewer-facing implication follows
+
+Weak draft behavior:
+
+- repeating several visible numbers in the paragraph
+- re-reading the plot in sentence form
+- spending prose budget on facts already obvious from the display
+
+Stronger oral behavior:
+
+- briefly anchor one or two numbers when needed
+- spend most of the prose on interpretation and mechanism
+- let the reader use the display for detail and the prose for understanding
+
+For data analysis specifically, good prose should identify:
+
+- the trend across settings
+- the governing tradeoff
+- the likely underlying cause
+- the true scope of the conclusion
+
+## 7. Separate experiments from analysis
+
+When the paper has enough material, split these functions:
+
+- `Experiments`: establish where the method wins and under what settings
+- `Analysis`: explain why the win is credible, what tradeoff it implies, and which objections have been addressed
+
+This split is often the difference between "result dump" and "mature paper."
+
+## 8. Use analysis to answer reviewer objections
+
+Analysis should not just repeat results in prose. It should answer questions such as:
+
+- Is the gain only visible under one summary metric?
+- Is the method simply more conservative?
+- Does the result survive baseline retuning?
+- Does the behavior transfer across models or tasks?
+- Is there a concrete example showing the claimed mechanism?
+
+Useful analysis blocks:
+
+- diversity versus accuracy
+- efficiency or runtime
+- tuned-baseline comparisons
+- transfer checks
+- qualitative examples
+- failure cases
+- case studies with mechanism-level interpretation
+
+When possible, analysis paragraphs should read as causal or mechanistic explanations, not as spreadsheet narration.
+
+Weak analysis often fails in a subtler way:
+
+- it cites the right values
+- it points at the right figure
+- but it never says what stable pattern the reader should infer or what that pattern reveals about the method
+
+## 9. Memorable case studies matter
+
+Aggregate metrics build confidence. Case studies build recall.
+
+A strong case study often includes:
+
+- one concrete prompt or instance
+- competing outputs or behaviors
+- the exact error or success mode
+- a visualization or local statistic explaining the difference
+
+This gives reviewers something they can describe in conversation.
+
+## 10. Page pacing is part of readability
+
+A draft can be technically correct and still feel rushed because every page is overloaded.
+
+Audit pacing by asking:
+
+- Does this page have one dominant job?
+- Is the key display large enough to read comfortably?
+- Are several local claims competing for the same visual space?
+- Would moving one result block to appendix improve the argument?
+
+Oral-quality papers leave breathing room for important evidence.
+
+## 11. Appendix is part of the package
+
+The appendix should feel designed, not appended.
+
+Common appendix roles:
+
+- protocol and evaluation details
+- full benchmark tables
+- human evaluation protocol details
+- additional ablations or sweeps
+- failure cases and extra examples
+- proofs or synthetic illustrations
+- implementation and reproducibility details
+
+Main text should point to the appendix deliberately, not apologetically.
+
+## 12. Fast audit: draft signal to oral fix
+
+- `Overloaded abstract` -> restore hierarchy: problem, idea, main pattern, one calibrated takeaway
+- `Introduction jumps to wins` -> add broader framing and early intuition anchor
+- `Method is correct but cold` -> add conceptual contrast and interpretation
+- `Results read like spreadsheets` -> separate pattern, key numbers, interpretation
+- `Analysis paraphrases the display` -> keep only the few anchoring facts needed, then explain why the pattern appears
+- `Analysis lists numbers but not the trend` -> name the stable pattern, the tradeoff, and the likely mechanism
+- `Analysis feels optional` -> promote at least one interpretive block into the main text
+- `Figures summarize but do not teach` -> add or redesign an intuition figure and a case-study figure
+- `Main text is crowded` -> move exhaustive material to appendix
+- `Method defense is too thick` -> compress repeated formal defense and spend that budget on objection handling
+- `Appendix is just tables` -> organize it around protocol, full results, stress tests, examples, failure cases, and reproducibility
+- `Claim wording outruns the data` -> narrow to the strongest evidence zone
+- `Artifact availability is inconsistent` -> make abstract, main text, reproducibility, and appendix say the same thing
+
+## 13. What to preserve from the stronger version
+
+When comparing two versions of a paper, preserve changes that improve:
+
+- onboarding
+- display roles
+- prose-display division of labor
+- analysis depth
+- trend extraction and mechanism explanation
+- page pacing
+- reviewer concern coverage
+- appendix intentionality
+
+Do not preserve changes only because they add length. Preserve them because they make the paper easier to trust, remember, and defend.
+
+## 14. Sectioned generation pattern
+
+When the manuscript is generated in sections instead of one pass, a strong default order is:
+
+1. plan
+2. introduction
+3. related work
+4. method
+5. experiments
+6. analysis
+7. appendix
+8. limitations
+9. conclusion
+10. abstract
+11. integration
+
+This order works because:
+
+- early sections establish the story and novelty boundary
+- experiments and analysis are separated before the abstract is finalized
+- appendix support exists before scope and limitation language is frozen
+- the integration pass can repair cross-section drift without reopening the entire writing problem

package/src/skills/write/references/oral_writing_principles.md

@@ -0,0 +1,291 @@
+# Oral Writing Principles
+
+This reference distills the difference between a strong draft and a top-conference oral paper.
+
+## 1. Optimize for reviewer cognition
+
+A draft often tries to say everything. An oral paper tries to make the reader:
+
+- understand the core idea quickly
+- remember the main claim
+- trust the evidence
+- repeat the paper's contribution accurately
+
+Write for cognition, not for compression.
+
+## 2. Build a story spine before polishing prose
+
+Before line editing, define:
+
+- the problem
+- the deficiency in current methods
+- the core idea
+- why the idea is principled
+- the main empirical pattern
+- the key mechanism or explanation
+- the exact scope of the claim
+
+If this spine is weak, local prose polish will not fix the paper.
+
+## 3. Section jobs
+
+Each main section should have one dominant function:
+
+- `Abstract`: compress the whole paper into one coherent impression
+- `Introduction`: motivate, frame the gap, preview the contribution, and make the reader care
+- `Related Work`: position the paper early enough that novelty is legible
+- `Method`: define and defend the method
+- `Experiments`: establish the main empirical pattern
+- `Analysis`: explain why the pattern is credible and important
+- `Conclusion`: restate contribution, scope, and implication without overselling
+
+When a section tries to do too many jobs, it usually becomes dense and forgettable.
+
+## 4. Paragraph jobs
+
+A paragraph should usually do one main thing:
+
+- set up a problem
+- state a pattern
+- interpret a figure or table
+- answer an objection
+- explain a mechanism
+
+If a paragraph simultaneously motivates, reports numbers, argues novelty, and calibrates claims, split it.
+
+## 5. Signposting is mandatory
+
+Use explicit transitions when changing jobs. Good signals include:
+
+- `To address this deficiency`
+- `The key question is`
+- `This result matters because`
+- `A reasonable concern is`
+- `This plot shows`
+- `This contrast illustrates`
+- `Taken together`
+
+These phrases are not fluff. They reduce the reader's organizational burden.
+
+## 6. Method sections must teach, not only define
+
+A weak method section gives equations and maybe a theorem.
+
+A strong method section also provides:
+
+- intuition
+- conceptual contrast with alternatives
+- why the method should work
+- what property makes it different
+- what interpretation the reader should keep in mind while reading results
+
+The goal is not just mathematical correctness. The goal is reader uptake.
+
+## 7. Results are not a spreadsheet
+
+Do not write results as a raw sequence of numbers.
+
+Use a three-layer structure:
+
+1. state the main pattern
+2. anchor the pattern with key numbers
+3. interpret why those numbers matter
+
+When possible, separate:
+
+- main empirical win
+- robustness evidence
+- objection-handling evidence
+- mechanism evidence
+
+When writing data analysis, do not stop at "what value is larger." Push to:
+
+- what trend persists across settings
+- what tradeoff the method is managing
+- what mechanism best explains the pattern
+- what narrower claim is actually justified by that pattern
+
+## 8. Figures and tables need surrounding prose
+
+Every important display should be introduced and closed.
+
+Before the display, say:
+
+- why it appears here
+- what question it answers
+
+After the display, say:
+
+- the main takeaway
+- what the reader should remember
+
+Do not force the figure or table to do all the explanatory work alone.
+
+Do not waste the surrounding prose by re-reading the display line by line. The display should carry the detailed values; the prose should carry the interpretation.
+
+## 9. Use figures as narrative anchors
+
+Top papers use figures for different jobs:
+
+- an early intuition figure
+- a main result figure
+- a mechanism or case-study figure
+
+The most important figure often appears early, before the full technical development, to give the reader a visual handle on the idea.
+
+## 10. Keep the main text legible
+
+Main text is for:
+
+- the story
+- the key evidence
+- the interpretation
+
+Appendix is for:
+
+- full result tables
+- extra ablations
+- detailed proofs
+- extended examples
+- extra implementation details
+
+Do not let the main text become a storage container for every supporting number.
+
+## 11. Calibrate claims structurally
+
+Strong claim calibration is not just one sentence saying "we do not claim X."
+
+Claim calibration should appear in:
+
+- the chosen wording
+- the selected comparisons
+- the scope of conclusions
+- the discussion of limitations
+- the organization of evidence
+
+If the evidence only supports robustness under certain settings, say that clearly and build the section around that scope.
+
+Prefer wording that stays inside the strongest evidence zone:
+
+- `strong default`
+- `wins or ties most settings`
+- `more robust under sweep`
+
+Be careful with wording that implies strict dominance unless the evidence package really supports it.
+
+## 12. Remove LLM-draft signals
+
+Common LLM-like writing signals:
+
+- excessive density
+- summary-first writing with weak staging
+- low paragraph differentiation
+- late positioning
+- tables carrying too much of the paper
+- result sections that feel like narration of a spreadsheet
+- analysis that feels appended rather than integral
+
+Replace them with:
+
+- staged narrative
+- stronger signposting
+- section discipline
+- figure/table role clarity
+- mechanism-aware analysis
+
+## 13. The target feeling
+
+The final paper should feel like:
+
+- the authors know exactly what the contribution is
+- the evidence is in the right order
+- the reader is being guided, not tested
+- the paper is easy to summarize after one read
+
+That is what often separates a strong draft from an oral-quality paper.
+
+## 14. Evidence budget signals preparedness
+
+Oral papers usually look prepared for reviewer follow-up.
+
+That feeling comes from visible evidence allocation:
+
+- enough main-text space for the central claim
+- enough analysis to explain the claim
+- enough appendix support to answer deeper questions
+
+If every display is overloaded or every concern is answered in one sentence, the paper will still feel underdeveloped even if the raw evidence exists.
+
+Budget allocation can also fail when too much space is spent defending the method repeatedly while too little space is spent answering the objections reviewers are actually likely to raise.
+
+## 15. Separate experiments from analysis when the paper can support it
+
+If the draft has enough material, use different sections for different jobs:
+
+- `Experiments` to establish the pattern
+- `Analysis` to explain, defend, and interpret the pattern
+
+This often improves pacing and makes the paper feel more deliberate.
+
+## 16. Memorable evidence beats only aggregate evidence
+
+Aggregate tables and plots establish scale.
+
+But oral papers also benefit from at least one memorable, concrete piece of evidence:
+
+- a case study
+- a failure mode
+- a qualitative contrast
+- a mechanism-level visualization
+
+This is what makes the paper easy to discuss after one reading.
+
+## 17. Main-text prose is for explanation, not duplication
+
+Main-text prose is scarce and expensive.
+
+Use it to answer:
+
+- why this result appears
+- why this tradeoff matters
+- why a reviewer should trust the interpretation
+- what mechanism connects the display to the claim
+
+For data analysis, the prose should surface the trend and essence of the result, not just the surface comparison.
+
+Do not spend it on long numeric narration when the table or figure already shows the facts.
+
+## 18. Page pacing matters
+
+Even a correct paper can feel draft-like if every page is trying to win too many arguments.
+
+Prefer:
+
+- one dominant job per page or subsection
+- readable figure sizes
+- visible transitions between result blocks
+- moving exhaustive material to appendix instead of shrinking everything
+
+Compression is not the same thing as sophistication.
+
+## 19. Appendix should feel designed
+
+For oral-level work, appendix is part of the persuasive system.
+
+Use it to house:
+
+- full result matrices
+- protocol details
+- tuned baselines and sensitivity checks
+- failure cases
+- extra ablations
+- extended examples
+- proofs and formal details
+
+The appendix should make the paper look more complete, not more improvised.
+
+## 20. Artifact status must be globally consistent
+
+If the abstract says code is available, the reproducibility section, appendix, and any artifact note must say the same thing.
+
+Small inconsistencies about release status, anonymous repos, or availability dates create avoidable trust friction.