@research-copilot/plugin 1.1.17 → 1.1.19

package/dist/skills/idea-evaluator/references/paradigm-elephant.md

@@ -0,0 +1,112 @@
+# Elephant-in-the-Room hunting
+
+## Table of contents
+
+1. Core idea
+2. Technique A: real-world pain-point extraction
+3. Technique B: corner-case cataloguing
+4. Worked example: large-scale schema linking
+5. Worked example: multi-agent collaboration for complex tasks
+6. What to ignore
+
+## 1. Core idea
+
+Every research community has problems it knows are important but
+systematically avoids. Sometimes the problems are too hard for the
+current toolset. Sometimes they do not fit the publication conventions.
+Sometimes everyone assumes someone else will eventually solve them. If
+a researcher can formalise one of these elephants into a tractable
+academic problem, the reframing often becomes a new subfield.
+
+The hunting technique is to look outside the academic-benchmark bubble
+and inside the corner cases where current methods fail silently.
+
+## 2. Technique A: real-world pain-point extraction
+
+### Procedure
+
+1. Identify three practitioners (industry engineers, deployed-system
+   operators, or domain experts) who use the field's outputs in
+   production.
+2. Ask them: what is the single biggest pain-point they hit every
+   week that the academic literature does not address?
+3. For each pain-point, ask whether it is a research problem or an
+   engineering detail.
+4. For research-grade pain-points, ask: can this be formalised as a
+   benchmark, a new problem setting, or a measurable task?
+
+### Signals a pain-point is research-grade
+
+- Multiple practitioners independently raise it.
+- Current academic work implicitly assumes the problem does not
+  exist.
+- A rigorous formalisation would require new evaluation metrics, not
+  just a new dataset in an existing format.
+- The pain-point has been stable for at least two years; it is not a
+  passing artefact of one tool version.
+
+## 3. Technique B: corner-case cataloguing
+
+### Procedure
+
+1. Run the field's current best method on inputs drawn from the long
+   tail of real-world data.
+2. Catalogue the failure modes: classify each failure by root cause.
+3. Identify failure modes that are common, systematic, and not
+   addressed in the literature.
+4. For each, ask: is this a Prompt-Engineering patch problem, or a
+   fundamental limitation of the current paradigm?
+5. For fundamental limitations, propose a reframing that addresses
+   the root cause.
+
+### Signals a corner case is an elephant
+
+- Frequent: appears in at least 10 percent of real-world inputs.
+- Systematic: the same class of input consistently fails.
+- Unaddressed: searching the literature surfaces no direct work on
+  this failure mode.
+- Root-level: quick fixes do not stick; each patch exposes the next
+  layer of the same failure.
+
+## 4. Worked example: large-scale schema linking
+
+- **Academic benchmark focus**: Text-to-SQL evaluation on single-
+  table or small-schema problems with clean column names.
+- **Practitioner pain-point**: real databases have thousands of
+  tables with inconsistent or cryptic naming; even DBAs do not know
+  which tables to use for a given question.
+- **Formalisation**: large-scale schema linking or schema retrieval
+  as a first-class research problem, with its own benchmarks and
+  methods.
+- **Impact**: an entire subfield of retrieval-augmented Text-to-SQL
+  opens up, previously invisible under the clean-schema assumption.
+
+## 5. Worked example: multi-agent collaboration for complex tasks
+
+- **Academic focus**: single-agent LLMs judged on short, well-bounded
+  tasks.
+- **Practitioner pain-point**: complex real business workflows need
+  coordination across specialist roles; single agents spiral into
+  long chains, hallucinations, or loops.
+- **Corner-case analysis**: failure is not random; it correlates with
+  tasks requiring sustained state, cross-function expertise, or
+  rollback recovery.
+- **Formalisation**: multi-agent collaboration frameworks or Standard
+  Operating Procedure driven agent workflows, each with their own
+  evaluation protocols.
+- **Impact**: a new class of papers around agent orchestration rather
+  than agent intelligence.
+
+## 6. What to ignore
+
+- **Engineering pain-points**: "my deployment tool is slow" is
+  valuable to the practitioner but is not research-grade.
+- **One-off failures**: a single viral bug is not an elephant.
+- **Already-formalised problems**: if the literature already has a
+  subfield for this, the elephant has been captured. Move on.
+- **Pain-points that are solvable by the next model release**: if
+  scaling alone is expected to solve it, the reframing is premature.
+- **Elephants outside the user's reach**: some elephants require
+  institutional infrastructure (industry partnerships, proprietary
+  data) a PhD student cannot access alone. Record on the Hamming's
+  list and defer.

package/dist/skills/idea-evaluator/references/paradigm-examples.md

@@ -0,0 +1,107 @@
+# Canonical disruptive examples
+
+## Table of contents
+
+1. How to read these examples
+2. Example 1: Transformer architecture
+3. Example 2: Adaptive Query Processing
+4. Example 3: Agent-driven data cleaning
+5. Example 4: Multi-agent collaboration workflows
+6. Lessons across examples
+
+## 1. How to read these examples
+
+Each example below traces a published or widely-adopted line of work
+back to the disruptive reframing at its root. For each example, the
+relevant principle (First Principles, Elephant in the Room,
+Technology Cycle, Hamming's Rule) is identified. The purpose is to
+show what a disruptive reframing looks like in the wild and to
+calibrate paradigm-shift analysis against real precedents.
+
+Examples 2, 3, and 4 appear in
+`handbook/02_Idea_Generation/2.3_进阶_如何做颠覆式创新.md` as
+illustrations for the principle they anchor.
+
+## 2. Example 1: Transformer architecture
+
+- **Context**: circa 2017, the machine-translation community's
+  strongest models used complex CNN or RNN/LSTM architectures with
+  attention as a secondary mechanism.
+- **Consensus assumption**: sequential recurrence is required to
+  model long-range dependencies.
+- **Reframing (First Principles)**: what if attention alone is
+  sufficient? The Google team dropped all recurrence and offered a
+  pure-attention architecture, the Transformer.
+- **Outcome**: the basis of essentially all subsequent large language
+  models and the dominant architecture across NLP, vision, and
+  speech.
+- **Principle**: First Principles (assumption audit on recurrence)
+  plus Technology Cycle (GPU compute enabled large all-attention
+  models).
+
+## 3. Example 2: Adaptive Query Processing
+
+- **Context**: traditional database query optimisation.
+- **Consensus assumption**: a query plan must be fully determined
+  before execution; the optimiser uses static statistics.
+- **Reframing (First Principles)**: what if statistics change during
+  execution? Let the plan adapt to observed intermediate cardinalities.
+- **Outcome**: a multi-decade research programme on runtime re-
+  optimisation, progressive cardinality estimation, parametric
+  query plans.
+- **Principle**: First Principles (assumption audit on static plans)
+  plus Elephant in the Room (production systems always saw plan
+  failures due to estimation errors; academic world avoided this).
+
+## 4. Example 3: Agent-driven data cleaning
+
+- **Context**: traditional data cleaning uses standalone scripts
+  (rules, heuristics, small models) independently of downstream use.
+- **Consensus assumption**: cleaning is a preprocessing step
+  decoupled from downstream tasks.
+- **Reframing (Return to Purpose)**: the goal of cleaning is correct
+  downstream analysis. Let an LLM agent observe downstream task
+  performance and iteratively rewrite cleaning strategies.
+- **Outcome**: agent-driven data cleaning becomes a research
+  direction separate from classical cleaning; task-conditioned
+  cleaning collapses the preprocessing step into a feedback loop.
+- **Principle**: First Principles (return-to-purpose) plus Technology
+  Cycle (LLM agents make the feedback loop tractable).
+
+## 5. Example 4: Multi-agent collaboration workflows
+
+- **Context**: single-agent LLM systems for complex business tasks.
+- **Corner-case observation**: single agents loop, hallucinate, or
+  spiral on multi-role, long-horizon tasks such as building a full
+  web application end-to-end.
+- **Consensus assumption**: more prompt engineering or longer
+  context will fix multi-step failures.
+- **Reframing (Elephant in the Room)**: single-agent failure is not
+  a prompt-tuning problem; it is a coordination problem. Design a
+  multi-agent framework with role separation and standard operating
+  procedures.
+- **Outcome**: a research programme around agent orchestration,
+  multi-agent frameworks, agentic workflows, coordination protocols.
+- **Principle**: Elephant in the Room (avoided hard problem of long-
+  horizon multi-role tasks) plus Technology Cycle (LLM agent
+  availability).
+
+## 6. Lessons across examples
+
+- All four examples combine two of the four principles. Single-
+  principle reframings are often weaker; the strongest disruptive
+  work triangulates between assumptions, pain-points, technology
+  shifts, and important problems.
+- Three of the four required a Technology Cycle element to be
+  actionable. Without the shift, the reframing is either premature
+  (the technology does not yet exist) or too late (the community has
+  already absorbed it).
+- None of the four is a one-paper contribution. Each opened a
+  research programme that spanned years. A paradigm-shift analysis
+  should aim for programme-scale reframings, not individual-paper
+  tweaks.
+- All four were risky at the time. The Transformer paper faced
+  scepticism; Adaptive Query Processing was initially dismissed as
+  impractical; agent-driven cleaning is still debated; multi-agent
+  frameworks are still proving their value. Disruptive work lives
+  with reception risk.

package/dist/skills/idea-evaluator/references/paradigm-first-principles.md

@@ -0,0 +1,115 @@
+# First Principles thinking
+
+## Table of contents
+
+1. Core idea
+2. Technique A: assumption audit
+3. Technique B: return to purpose
+4. Worked example: Adaptive Query Processing
+5. Worked example: Agent-driven data cleaning
+6. Common misuses
+
+## 1. Core idea
+
+Do not start from the literature. Start from the physical or
+mathematical essence of the problem. Existing solutions inherit
+assumptions from the moment they were designed, and those assumptions
+may have quietly stopped being true. First Principles thinking
+strips a problem back to what is intrinsically required and asks
+whether the path from requirement to current solution still holds.
+
+The technique is deceptively simple: identify an assumption, test
+whether it still holds, and, if it does not, propose a new solution
+that does not require it.
+
+## 2. Technique A: assumption audit
+
+### Procedure
+
+1. Write down the three most widely-held assumptions in the subfield.
+   State each as a single declarative sentence.
+2. For each assumption, write its original justification (why was it
+   adopted in the first place?).
+3. For each assumption, ask what has changed since that justification
+   was established.
+4. Rank the assumptions by fragility. An assumption whose justification
+   depends on a condition that no longer holds is a candidate.
+5. Pick the most fragile assumption and imagine a system that does
+   not make it. What becomes possible?
+
+### What a good assumption looks like
+
+- Specific. "Data distribution is static" is better than "things are
+  fixed".
+- Historical. The audit names when and why the assumption was
+  adopted.
+- Testable. You can point to a concrete paper or system that still
+  relies on it.
+- Fragile. A recent technology or context shift weakens the
+  justification.
+
+## 3. Technique B: return to purpose
+
+### Procedure
+
+1. Ask: what is the user-facing goal of this task? Not the metric,
+   the goal.
+2. Ask: what are the intermediate steps the field has built to reach
+   the goal?
+3. Ask: for each intermediate step, is it required, or is it an
+   accident of how the field grew?
+4. Collapse any intermediate step that is not strictly required and
+   ask what the result looks like.
+
+### What return-to-purpose surfaces
+
+- Steps that exist because they were needed under the old hardware
+  regime, but are not needed now.
+- Steps that exist because the original tools could only produce
+  outputs at a certain level of abstraction.
+- Steps that exist because specialist roles historically split the
+  task.
+
+## 4. Worked example: Adaptive Query Processing
+
+- **Assumption**: "Database query plans must be fully determined
+  before execution." Justification: the cost model expects static
+  statistics and static workloads. Originally adopted in the 1970s.
+- **What changed**: workloads and data distributions became dynamic;
+  modern storage can observe intermediate results cheaply.
+- **Reframing**: what if the query plan can change during execution
+  based on observed data? This breaks the static-plan assumption and
+  creates the Adaptive Query Processing research programme.
+- **Impact**: decades of downstream work on runtime re-optimisation,
+  parametric query plans, learned cardinality.
+
+## 5. Worked example: Agent-driven data cleaning
+
+- **Goal revisit**: the actual goal is to make downstream analysis or
+  modelling correct. Data cleaning is an intermediate step.
+- **Step audit**: traditional cleaning writes scripts independent of
+  the downstream task. This was necessary when downstream tasks could
+  not observe cleaning decisions.
+- **Reframing**: what if an LLM agent observes the downstream
+  task's feedback and iteratively rewrites cleaning strategies? The
+  intermediate step collapses into a closed loop with the downstream
+  task.
+- **Impact**: Agent-driven Data Cleaning becomes a new direction
+  rather than a tweak to existing cleaning pipelines.
+
+## 6. Common misuses
+
+- **Assumption theatre**: naming an assumption and pretending to
+  break it, while the reframing still secretly relies on the same
+  assumption. Check whether your reframing depends on the assumption
+  anywhere in its pipeline.
+- **Straw assumptions**: auditing assumptions that nobody in the
+  field still holds. The audit should target live consensus, not
+  historical views.
+- **Unbounded reframing**: proposing a reframing that requires
+  technology that does not exist yet. If the reframing needs
+  something we will have in five years, record it on the Hamming's
+  Rule list and move on; do not treat it as actionable today.
+- **Skipping the justification audit**: if you cannot state why the
+  assumption was originally adopted, you cannot judge whether the
+  justification still holds.

package/dist/skills/idea-evaluator/references/paradigm-hamming.md

@@ -0,0 +1,137 @@
+# Hamming's Rule: the personal top-10 problem list
+
+## Table of contents
+
+1. Origin and why it matters
+2. Building the list
+3. The prepared-mind practice
+4. How often to refresh
+5. Sustained-problem selection
+6. The template
+
+## 1. Origin and why it matters
+
+Richard Hamming, in the talk "You and Your Research", argued that
+every researcher should carry a running list of the 10 to 20 most
+important problems in their field, regardless of whether the
+researcher currently has any idea how to solve them. The list is the
+scaffolding for prepared-mind thinking: when a new technique, paper,
+or idea crosses the researcher's desk, the first question becomes,
+"does this unlock one of my top-10 problems?"
+
+Without the list, novel techniques get absorbed into incremental work
+on whatever the researcher happens to be doing. With the list, novel
+techniques route toward the highest-leverage targets the researcher
+has pre-committed to.
+
+## 2. Building the list
+
+### Initial pass
+
+1. Brainstorm 15-20 candidate problems in the researcher's broad
+   area. Ignore solvability; include problems the researcher has no
+   current approach for.
+2. Cut the list to 10-12 by applying three filters:
+   - **Importance**: would solving this change the subfield in a
+     non-trivial way?
+   - **Durability**: will this problem still be important in five
+     years, or is it a passing artefact of current tooling?
+   - **Personal connection**: does the researcher care about this
+     problem enough to think about it while walking or showering?
+3. Write each surviving problem as one declarative sentence, not a
+   vague phrase.
+
+### Good problems on the list
+
+- Each one is specific enough that progress is recognisable.
+- Each one is important enough that an advisor would nod.
+- Each one is unsolvable by today's methods (at least not obviously).
+- Each one is in the subfield the researcher works in, not adjacent
+  subfields they do not know well.
+
+### Problems that should not be on the list
+
+- "Make LLMs better". Too vague.
+- "Publish three papers this year". Outcome goal, not problem.
+- "Beat GPT-4 on MMLU". Metric chasing, not a durable problem.
+- "Replicate paper X with my own code". Exercise, not research.
+
+## 3. The prepared-mind practice
+
+Once the list exists, the daily practice is:
+
+- Every time a new paper, algorithm, or technology crosses the
+  researcher's desk, ask: does this help with any problem on the
+  list?
+- Keep a running notebook of matches. Most matches will be false
+  positives, but occasionally a match is real.
+- When a real match appears, it is usually obvious in retrospect; the
+  technique unlocks a step in a problem the researcher has already
+  been mulling.
+- Dedicate at least a few hours a week to working on top-three
+  problems directly, even if they seem hopeless.
+
+## 4. How often to refresh
+
+- Every six months, review the list.
+- Cross off problems that have been solved by the community.
+- Add new problems that have emerged.
+- Move problems up or down based on current field pace.
+- Delete problems the researcher no longer cares about.
+
+The list should feel alive. A list that has not changed in two years
+is either remarkable foresight or a signal that the researcher has
+stopped paying attention to the field.
+
+## 5. Sustained-problem selection
+
+A crucial rule: do not only work on problems the researcher knows how
+to solve. Those problems produce incremental papers (which are
+valuable, see `references/five-dimensions.md` in the `idea-evaluator`
+skill) but they will not produce landmark work.
+
+Reserve 10-20 percent of research time for a top-five problem the
+researcher does not know how to solve. Most attempts will fail
+visibly. Some will make partial progress. One in 10 might crack open
+and define the researcher's career.
+
+Avoid the opposite failure mode: spending 100 percent of time on
+unsolvable top-10 problems with no incremental output. That produces
+an unpublishable PhD. The balance is roughly 70-20-10 between
+incremental, broader, and disruptive work (see the portfolio section
+of `idea-evaluator`'s `paradigm-shift-probe.md`).
+
+## 6. The template
+
+```markdown
+# My top-10 problems, as of <YYYY-MM-DD>
+
+1. <single-sentence problem statement>. Last touched: <date or never>.
+2. ...
+3. ...
+4. ...
+5. ...
+6. ...
+7. ...
+8. ...
+9. ...
+10. ...
+
+## Problems added this refresh
+
+- ...
+
+## Problems removed this refresh
+
+- ... (reason: solved, obsolete, or no longer personally interesting)
+
+## This month's attention allocation
+
+- Top-three problem I spent the most hours on: #<n>
+- Matches from other-field papers that seemed promising: <list>
+```
+
+Keep this file under version control. The history is a record of how
+the researcher's taste evolved, and it is surprising how often an
+early entry, seemingly unconnected at the time, turns out to anchor a
+late-career project.

package/dist/skills/idea-evaluator/references/paradigm-shift-probe.md

@@ -0,0 +1,160 @@
+# Paradigm-shift probe
+
+## Table of contents
+
+1. Incremental versus disruptive
+2. The four probing principles
+3. Scoring the probe
+4. When to delegate to handbook 2.3
+5. Risk and reward trade-off
+6. Portfolio strategy for a PhD
+
+## 1. Incremental versus disruptive
+
+Most published ideas are incremental: they take an existing baseline
+and push it on one or two of the Higher, Faster, Stronger, Cheaper,
+Broader dimensions. Incremental work is the backbone of a PhD career
+and can absolutely reach top venues.
+
+Disruptive work reshapes the problem itself rather than its solution
+it challenges a hidden assumption, surfaces a previously ignored
+problem, or rides a technology-cycle shift that makes a new approach
+possible. Disruptive ideas carry higher reward (a strong disruptive
+paper defines the subfield for years) and higher risk (most disruptive
+attempts fail or take years to pay off).
+
+The probe is not a gate. Even a strictly incremental idea can justify
+a top-venue paper. The probe's purpose is twofold:
+
+- Identify disruptive potential early, so the student can choose to
+  pursue it consciously rather than accidentally reducing it to
+  incremental work mid-project.
+- Flag ideas that look disruptive but are actually incremental (or
+  vice versa), so the Introduction framing matches the reality.
+
+## 2. The four probing principles
+
+### 2.1 First Principles
+
+Does the idea challenge an assumption the field takes for granted?
+
+Signal that yes: the idea starts with "everybody assumes X, but what
+if X is wrong" and produces a specific X. Examples of X include
+assumptions like "iterative methods require full-dataset inference
+every round" (LEAD broke this), or "visualisation quality reduces to
+aesthetics or correctness alone" (VisJudge-Bench broke this).
+
+Signal that no: the idea starts with a known technique and proposes a
+variant. No assumption is challenged, only a hyperparameter or a
+training recipe is tweaked.
+
+### 2.2 Elephant in the Room
+
+Does the idea address a problem the community sees but avoids?
+
+Signal that yes: the paper includes a section that says "this problem
+has been known for years but nobody has touched it because X", and
+the paper provides a path past X. Examples: agent evaluation beyond
+toy tasks; reproducibility of RLHF across labs; domain shift in
+production deployment.
+
+Signal that no: the problem is new and small, rather than old and
+large.
+
+### 2.3 Technology Cycle
+
+Does a recent technology shift enable this idea to work now, when it
+could not before?
+
+Signal that yes: the idea depends on capability that did not exist
+two years ago. Example: LLM agents only became practical after
+instruction-tuned base models matured; reinforcement-learning from
+execution feedback only became tractable after cheap-inference
+infrastructure. An idea that could have been done five years ago but
+was not, is usually incremental.
+
+Signal that no: the idea could have been attempted at any point in
+the last decade with minor modifications.
+
+### 2.4 Hamming's Rule
+
+If this idea succeeded, would it change the community's top priority
+list?
+
+Signal that yes: the idea is a top-three problem in the student's
+advisor's list; colleagues light up when they hear it; reviewers would
+call it a landmark paper if done well.
+
+Signal that no: the idea is niche, or its success improves a metric
+that only a handful of people care about.
+
+## 3. Scoring the probe
+
+Answer each of the four principles as Yes, Partial, or No. Count Yes
+as 2, Partial as 1, No as 0. Total score 0-8.
+
+- **0-2**: pure incremental. Frame as incremental in the Introduction;
+  emphasise Higher or Faster.
+- **3-4**: incremental with disruptive seeds. Explicitly call out the
+  disruptive aspect in the Introduction to capture reader attention.
+- **5-6**: disruptive potential. Consider reading handbook 2.3 to
+  deepen the framing.
+- **7-8**: genuine paradigm-shift candidate. Definitely study handbook
+  2.3; the Introduction should lead with First Principles framing.
+
+## 4. When to delegate to handbook 2.3
+
+Recommend the user read handbook 2.3 when:
+
+- Probe score is 5 or higher.
+- Probe score is lower but the user explicitly wants to pursue a
+  disruptive angle.
+- The idea has a First Principles signal with a concrete hidden
+  assumption that the user can state in one sentence.
+- The paper is targeting a venue that rewards paradigm-shift framing
+  (ICLR Outstanding, NeurIPS Spotlight, VLDB Best Paper).
+
+Do not delegate when the idea is genuinely incremental; forcing
+disruptive framing on incremental work weakens the paper.
+
+## 5. Risk and reward trade-off
+
+Disruptive work has three distinct risk categories:
+
+- **Execution risk**: the idea is right but the experiments fail. Higher
+  for disruptive because the mechanism is unproven.
+- **Reception risk**: the idea is right and the experiments succeed,
+  but reviewers do not recognise the contribution. High for early
+  disruptive work.
+- **Timeline risk**: the idea requires more iterations than an
+  incremental project; lifecycle fit tightens.
+
+Against these, the rewards of a successful disruptive paper:
+
+- Career-defining impact on a single paper.
+- A new research programme around the framing.
+- Citation compounding over 5+ years.
+
+Rule of thumb: if the student has already published at least one
+incremental top-venue paper, they have earned the credibility to
+attempt a disruptive project. If not, anchor the first paper in
+incremental territory and let disruptive come second.
+
+## 6. Portfolio strategy for a PhD
+
+A healthy PhD research portfolio across 3-5 years usually has:
+
+- 60-70 percent incremental work: steady publications, builds the
+  student's technical depth and writing muscle.
+- 20-30 percent cross-disciplinary or Broader work: transplants a
+  known idea to a new context; lower risk than disruptive, higher
+  reward than incremental.
+- 10-20 percent disruptive work: one or two ambitious projects over
+  the PhD; not every one will land, but a single success pays for
+  several failed attempts.
+
+Use the paradigm-shift probe to classify each candidate project and
+balance the portfolio explicitly. The worst outcome is four disruptive
+projects that all fail to publish; the second worst is four
+incremental projects that publish but fail to differentiate the
+student from their peers.