@research-copilot/plugin 1.1.17 → 1.1.19

package/dist/skills/tech-paper-template/references/thinking-template.md

@@ -0,0 +1,194 @@
+# Thinking template: cell-by-cell content contract
+
+## Table of contents
+
+1. How to read this reference
+2. Cell: Research background
+3. Cell: Limitations 1, 2, 3
+4. Cell: Key Idea or Our Goal
+5. Cell: Challenges 1, 2, 3
+6. Cell: Methodology topic sentence and modules
+7. Cell: Contributions
+8. Methodology-outline derivation
+
+## 1. How to read this reference
+
+Each cell in the thinking template has a content contract: what
+belongs in the cell, what a strong filling looks like, what a weak
+filling looks like, and how to detect trouble.
+
+The template is a thinking tool, not a drafting tool. Each cell
+should be fillable in one to four sentences. If a cell fights to
+expand beyond that, either the paper has too much ambition for one
+paper or the writer does not yet have clarity on that cell.
+
+## 2. Cell: Research background
+
+### Content contract
+
+One paragraph equivalent. Names the research scenario, explains why
+it matters in the real world, cites 3 to 5 recent representative
+works in the subfield.
+
+### Strong filling
+
+- Specific scenario (not "AI is important").
+- Real-world stakes named (a user, a deployed system, a decision
+  that changes).
+- Two or three canonical recent works cited by name.
+- Roughly 60 to 100 words.
+
+### Weak filling
+
+- Generic motivation.
+- "Deep learning has revolutionised ..." as the opening line.
+- No named prior works.
+
+### Detection rule
+
+If the filling does not name at least one specific prior work or
+one concrete real-world stakes element, the cell is weak.
+
+## 3. Cell: Limitations 1, 2, 3
+
+### Content contract
+
+At most three limitations. Each limitation is a specific gap in
+prior work, framed as "prior work X does not handle Y".
+
+### Strong filling
+
+- Each limitation names a specific capability.
+- Each limitation corresponds to a challenge or design decision
+  later in the template.
+- Two limitations is acceptable; do not pad.
+
+### Weak filling
+
+- "Existing work is insufficient".
+- Four or more limitations.
+- Limitations with no connection to any challenge.
+
+### Detection rule
+
+If any limitation does not map to a challenge, either the
+limitation is noise or the challenge is missing. Resolve before
+proceeding.
+
+## 4. Cell: Key Idea or Our Goal
+
+### Content contract
+
+One sentence. For Technique: the core mechanism (Key Idea). For
+New Problem/Setting: the goal or problem formulation.
+
+### Strong filling
+
+- A sentence a reviewer could quote.
+- Specific enough that the reader could re-derive the method
+  sketch.
+- Free of marketing language ("revolutionary", "breakthrough").
+
+### Weak filling
+
+- Vague phrases ("we propose a new method").
+- Multiple sentences trying to compress the whole paper.
+- Restating the research question without an answer.
+
+### Detection rule
+
+If the cell spans more than two sentences, the writer has not yet
+committed to a single framing. Force the choice.
+
+## 5. Cell: Challenges 1, 2, 3
+
+### Content contract
+
+At most three challenges. Each challenge is a specific obstacle
+that blocks a naive extension of prior work from solving the
+problem.
+
+### Strong filling
+
+- Each challenge is grounded in implementing the Key Idea or
+  realising the Goal.
+- Each challenge cites the specific mechanism-level difficulty.
+- Each challenge maps one-to-one to a methodology module.
+
+### Weak filling
+
+- "The problem is hard".
+- Challenges pre-announcing their solutions.
+- More than three challenges; the scope is too wide.
+
+### Detection rule
+
+If the challenge is stated without explanation of why a naive
+approach fails, the challenge is probably invented to justify a
+module. Re-derive from the Key Idea.
+
+## 6. Cell: Methodology topic sentence and modules
+
+### Content contract
+
+One topic sentence naming the framework or system plus one
+sub-cell per methodology module. Each module addresses one
+challenge.
+
+### Strong filling
+
+- Topic sentence names the framework and hints at the overall
+  mechanism in under 25 words.
+- Each module is named and has a one-sentence summary.
+- One-to-one correspondence with challenges.
+
+### Weak filling
+
+- Topic sentence is a cliche ("we propose a novel framework").
+- Modules without names ("a module").
+- Module count exceeds challenge count without explanation.
+
+### Detection rule
+
+If the module count and challenge count do not match, one side is
+wrong. Fix before proceeding.
+
+## 7. Cell: Contributions
+
+### Content contract
+
+3 or 4 numbered contributions. Each maps to a section.
+
+### Strong filling
+
+- Each contribution is specific and citable.
+- Each contribution is mapped to a section or set of sections.
+- Contributions cover the main mechanism, the main result, and
+  either the problem framing (New Problem) or the system design
+  (Technique).
+
+### Weak filling
+
+- "Extensive experiments".
+- "Comprehensive analysis".
+- Four-plus contributions.
+- Contributions without section mapping.
+
+### Detection rule
+
+Run the contribution-patterns audit from the `intro-drafter` skill's
+`references/contribution-patterns.md` (cited cross-skill in prose,
+not as a reference pointer).
+
+## 8. Methodology-outline derivation
+
+Once the template is filled, derive a methodology-section outline
+by:
+
+1. Using the methodology topic sentence as Section 3's opening.
+2. Making each module a numbered subsection.
+3. For each subsection, copying the module summary as the opening
+   sentence and listing three to five sub-points to expand.
+
+This skeleton is what the author uses to draft the methodology
+section; it is handed off to the writer after this skill completes.

package/dist/skills/tech-paper-template/references/worked-examples.md

@@ -0,0 +1,124 @@
+# Filled thinking templates
+
+## Table of contents
+
+1. How to read these examples
+2. Example A: Alpha-SQL, Technique
+3. Example B: AFlow, Technique with cross-domain framing
+4. Example C: LEAD, New Problem/Setting
+5. Cross-example observations
+
+## 1. How to read these examples
+
+Each example below is a retrospective filling of the thinking
+template for a published paper. Ratings reflect what a strong
+pre-submission filling looks like. Sources are
+`handbook/06_Case_Studies/6.1`, `6.2`, and
+`6.3`.
+
+## 2. Example A: Alpha-SQL, Technique
+
+### Paper-type positioning
+
+Technique Paper. Text-to-SQL is well-established; contribution is a
+new inference-time mechanism (MCTS search).
+
+### Filled template
+
+| Stage | Content |
+|---|---|
+| Research background | Text-to-SQL for enterprise analytics. Production use by analysts who do not know SQL. Recent work: OpenAI Codex, DIN-SQL, DailySQL |
+| Limitation 1 | Single-pass prompting fails on complex multi-table JOIN queries |
+| Limitation 2 | Fine-tuning on each schema is expensive and does not generalise |
+| Key Idea | Use Monte Carlo Tree Search at inference time to explore SQL candidates and score them via execution feedback |
+| Challenge 1 | SQL candidate space is combinatorial; naive search is intractable |
+| Challenge 2 | No trained critic is available for inference-time reward |
+| Methodology topic sentence | Alpha-SQL combines structured MCTS search over SQL parse trees with self-supervised reward from execution feedback |
+| Module A (Challenge 1) | MCTS-guided SQL generation with learned expansion priors |
+| Module B (Challenge 2) | Self-supervised reward via execution-based validation |
+| Contribution 1 | MCTS-based Text-to-SQL inference framework (Section 3) |
+| Contribution 2 | Self-supervised reward design (Section 4) |
+| Contribution 3 | Experiments on BIRD and Spider with N-point gains (Section 5) |
+
+### Consistency checks
+
+- Check 1 (Limitations -> Key Idea): pass. Both limitations addressed
+  by MCTS-at-inference.
+- Check 2 (Key Idea -> Challenges): pass. Both challenges arise from
+  MCTS.
+- Check 3 (Challenges -> Methodology): pass. One-to-one mapping.
+- Check 4 (Methodology -> Contributions): pass.
+
+## 3. Example B: AFlow, Technique with cross-domain framing
+
+### Paper-type positioning
+
+Technique Paper, Mixed cross-domain. Operator-graph search
+transplanted from NAS to agent workflow.
+
+### Filled template
+
+| Stage | Content |
+|---|---|
+| Research background | LLM agents for code generation; HumanEval and MBPP benchmarks; recent work on agent workflows |
+| Limitation 1 | Prompt engineering alone yields brittle workflows |
+| Limitation 2 | Single-agent architectures do not capture operator composition |
+| Limitation 3 | Hand-designed workflows do not generalise across task families |
+| Key Idea | Formulate agent-workflow design as search over operator graphs, borrowing from neural architecture search |
+| Challenge 1 | Operator-graph search space is combinatorial |
+| Challenge 2 | Reward signal for a workflow is discrete and sparse |
+| Methodology topic sentence | AFlow combines operator-graph search guided by execution-based rewards |
+| Module A (Challenge 1) | Operator-graph search with learned priors |
+| Module B (Challenge 2) | Structured reward via code execution |
+| Contribution 1 | Formulate workflow design as operator-graph search (Section 2) |
+| Contribution 2 | AFlow search algorithm (Section 3) |
+| Contribution 3 | Experiments on HumanEval, MBPP with consistent gains over hand-designed workflows (Section 4) |
+
+### Consistency checks
+
+- All four pass.
+- Cross-domain framing is reflected in Contribution 1.
+
+## 4. Example C: LEAD, New Problem/Setting
+
+### Paper-type positioning
+
+New Problem/Setting Paper. Goal carries the narrative.
+
+### Filled template
+
+| Stage | Content |
+|---|---|
+| Research background | Instruction tuning of LLMs; data quality beats quantity; recent work on iterative data selection |
+| Limitation 1 | Non-iterative selection does not adapt to model evolution |
+| Limitation 2 | Iterative methods require expensive full-dataset inference per round |
+| Key Idea / Our Goal | (Goal) Iterative data selection without any additional inference overhead. (Key Idea) Use the training loss already computed inside the fine-tuning loop as a zero-overhead utility signal |
+| Challenge 1 | Extract a reliable utility signal from noisy training-loss trajectories |
+| Challenge 2 | Integrate the selection signal into the training loop without disturbing convergence |
+| Methodology topic sentence | LEAD uses Instance-level Dynamic Uncertainty as a utility signal and defers selection to a cadence that preserves convergence |
+| Module A (Challenge 1) | Instance-level Dynamic Uncertainty signal with statistical denoising |
+| Module B (Challenge 2) | Deferred selection integrated into the fine-tuning loop |
+| Contribution 1 | Problem formulation: iterative data selection without additional inference (Section 2) |
+| Contribution 2 | Instance-level Dynamic Uncertainty with theoretical analysis (Sections 3, 4) |
+| Contribution 3 | LEAD framework design (Section 3) |
+| Contribution 4 | Extensive experiments (Section 5) |
+
+### Consistency checks
+
+- All four pass.
+- Contribution 1 directly encodes the Goal.
+
+## 5. Cross-example observations
+
+- All three examples use 2 or 3 limitations, 2 challenges, 2 methodology
+  modules. The consistency of these counts is not accidental.
+- All three examples pass the four consistency checks. This is the
+  minimum bar for a strong top-venue submission.
+- The New Problem/Setting example (LEAD) has four contributions
+  because the problem formulation takes one contribution slot,
+  leaving three for the method and experiments.
+- The Mixed case (AFlow) leads with the cross-domain framing in
+  Contribution 1. This is the canonical pattern for cross-domain
+  Technique papers.
+- None of the three examples contains a vague contribution. Every
+  contribution is specific and maps to a section.

package/dist/skills/training-check/SKILL.md

@@ -0,0 +1,83 @@
+---
+name: training-check
+description: "Interactively monitor training metrics from the current Codex session, periodically checking WandB or fallback logs for NaN, divergence, plateaus, and broken runs."
+argument-hint: [wandb-run-or-monitoring-brief]
+allowed-tools: Bash(*), Read, Write, Edit, Grep, Glob
+---
+
+# Training Check
+
+You are now in **interactive watch** / 交互式训练监控模式.
+
+Keep the current session open and report directly in the current terminal. The user is watching this terminal for updates. By default, run a training health check every 30 minutes, output a concise but complete analysis report after each check, state the next check time, then continue monitoring.
+
+This skill checks training **quality**, not basic process health. Process health checks such as whether a tmux session exists or whether the GPU is idle can be handled by watchdog-style tooling; this skill focuses on whether the run is still worth continuing.
+
+## Inputs To Establish First
+
+Before the first check, identify or ask for the minimum monitoring context:
+
+- WandB run path or URL, if available.
+- Fallback log path, SSH command, or local command for reading recent training logs.
+- Training target, expected baseline, and key metrics that define success.
+- How the training was launched, so it can be stopped if needed.
+- Project notes path for recording decisions and evidence.
+
+If a source is unavailable, say so clearly and continue with the available source. If both WandB and fallback logs are unreachable, report the connectivity issue, classify the round as `WAIT`, and check again later. Do not infer that training is bad only because data is unreachable.
+
+## Per-Round Check
+
+Every round, read WandB first when configured. If WandB is unreachable, read the fallback logs. Inspect at least:
+
+- Training loss trend over recent checkpoints or steps.
+- Eval metrics and whether they improve, flatten, or degrade against baseline.
+- NaN or Inf in loss, gradients, activations, or logged metrics.
+- Sudden loss spikes, divergence, or repeated failed evaluations.
+- Learning rate schedule behavior.
+- Gradient norm, if logged.
+- Plateau patterns that suggest the run is no longer useful.
+
+Output one report in the current terminal with this structure:
+
+```text
+## Training Check - <local timestamp>
+
+- Data source: wandb_ok | log_fallback | unreachable
+- Run: <wandb run or training identifier>
+- Recent metrics: <loss/eval/lr/grad summary>
+- Anomalies: <NaN/Inf/spike/divergence/plateau findings>
+- Evidence: <WandB URL, log lines, metric values, or files inspected>
+- Decision: CONTINUE | WAIT | STOP
+- Reason: <why this decision is justified>
+- Next check: <local timestamp, normally 30 minutes later unless ending>
+```
+
+Use the decisions as follows:
+
+| Decision | Meaning | Action |
+|----------|---------|--------|
+| `CONTINUE` | Run looks healthy enough to keep training. | Keep monitoring and check again in 30 minutes. |
+| `WAIT` | Evidence is inconclusive, noisy, too early, or temporarily unreachable. | Do not stop training; keep monitoring and check again later. |
+| `STOP` | Training is clearly problematic or no longer worth continuing. | Stop the training task, save evidence, write notes, output final summary, and end monitoring. |
+
+## Stop Behavior
+
+When the decision is `STOP`:
+
+- Stop the training task.
+- If the context contains `stop_command`, run `stop_command` first.
+- If no `stop_command` is available, choose the appropriate stop action from how the training was launched, such as stopping the relevant tmux session, local process, remote process, scheduler job, or notebook job.
+- Save evidence: WandB URL, key metrics, relevant log snippets, files inspected, and the reason for stopping.
+- Append a project note for debugging and future analysis.
+- Output `FINAL_SUMMARY` in the terminal.
+- End the interactive monitoring loop.
+
+Never stop on the first sign of ordinary metric noise. Look for sustained trends, hard failures, or clear divergence. Always preserve enough evidence for a later agent or human to understand why the run was stopped.
+
+## Interactive Loop Guidance
+
+- The normal interval is 30 minutes.
+- If a round is `CONTINUE`, announce the next check time and wait until then.
+- If a round is `WAIT`, explain what evidence is missing or noisy and check again later. Use a shorter interval only when the run looks suspicious but not yet stop-worthy.
+- If an anomaly recovers, say so explicitly and continue monitoring.
+- Keep the user-facing report short enough to read in a terminal, but include concrete metric values and evidence paths.