dsh-aris-panel 0.1.0

package/skills/skills-codex/research-refine-pipeline/SKILL.md

@@ -0,0 +1,186 @@
+---
+name: "research-refine-pipeline"
+description: "Run an end-to-end workflow that chains `research-refine` and `experiment-plan`. Use when the user wants a one-shot pipeline from vague research direction to focused final proposal plus detailed experiment roadmap, or asks to \"\u4e32\u8d77\u6765\", build a pipeline, do it end-to-end, or generate both the method and experiment plan together."
+---
+
+# Research Refine Pipeline: End-to-End Method and Experiment Planning
+
+Refine and concretize: **$ARGUMENTS**
+
+## Overview
+
+Use this skill when the user does not want to stop at a refined method. The goal is to produce a coherent package that includes:
+
+- a problem-anchored, elegant final proposal
+- the review history explaining why the method is focused
+- a detailed experiment roadmap tied to the paper's claims
+- a compact pipeline summary that says what to run next
+
+This skill composes two existing workflows:
+
+1. `research-refine` for method refinement
+2. `experiment-plan` for claim-driven validation planning
+
+For stage-specific detail, read these sibling skills only when needed:
+
+- `../research-refine/SKILL.md`
+- `../experiment-plan/SKILL.md`
+
+## Core Rule
+
+Do not plan a large experiment suite on top of an unstable method. First stabilize the thesis. Then turn the stable thesis into experiments.
+
+## Default Outputs
+
+- `refine-logs/FINAL_PROPOSAL.md`
+- `refine-logs/REVIEW_SUMMARY.md`
+- `refine-logs/REFINEMENT_REPORT.md`
+- `refine-logs/EXPERIMENT_PLAN.md`
+- `refine-logs/EXPERIMENT_TRACKER.md`
+- `refine-logs/PIPELINE_SUMMARY.md`
+
+## Workflow
+
+### Phase 0: Triage the Starting Point
+
+- Extract the problem, rough approach, constraints, resources, and target venue.
+- Check whether `refine-logs/FINAL_PROPOSAL.md` already exists and still matches the current request.
+- If the proposal is missing, stale, or materially different from the current request, run the full `research-refine` stage.
+- If the proposal is already strong and aligned, reuse it and jump to experiment planning.
+- If in doubt, prefer re-running `research-refine` rather than planning experiments for the wrong method.
+
+### Phase 1: Method Refinement Stage
+
+Run the `research-refine` workflow and keep its V3 philosophy intact:
+
+- preserve the Problem Anchor
+- prefer the smallest adequate mechanism
+- keep one dominant contribution
+- modernize only when it improves the paper
+
+Exit this stage only when these are explicit:
+
+- the final method thesis
+- the dominant contribution
+- the complexity intentionally rejected
+- the key claims and must-run ablations
+- the remaining risks, if any
+
+If the verdict is still `REVISE`, continue into experiment planning only if the remaining weaknesses are clearly documented.
+
+### Phase 2: Planning Gate
+
+Before the experiment stage, write a short gate check:
+
+- What is the final method thesis?
+- What is the dominant contribution?
+- What complexity was intentionally rejected?
+- Which reviewer concerns still matter for validation?
+- Is a frontier primitive central, optional, or absent?
+
+If these answers are not crisp, tighten the final proposal first.
+
+### Phase 3: Experiment Planning Stage
+
+Run the `experiment-plan` workflow grounded in:
+
+- `refine-logs/FINAL_PROPOSAL.md`
+- `refine-logs/REVIEW_SUMMARY.md`
+- `refine-logs/REFINEMENT_REPORT.md`
+
+Ensure the experiment plan covers:
+
+- the main anchor result
+- novelty isolation
+- a simplicity or deletion check
+- a frontier necessity check if applicable
+- run order, budget, and decision gates
+
+### Phase 4: Integration Summary
+
+Write `refine-logs/PIPELINE_SUMMARY.md`:
+
+```markdown
+# Pipeline Summary
+
+**Problem**: [problem]
+**Final Method Thesis**: [one sentence]
+**Final Verdict**: [READY / REVISE / RETHINK]
+**Date**: [today]
+
+## Final Deliverables
+- Proposal: `refine-logs/FINAL_PROPOSAL.md`
+- Review summary: `refine-logs/REVIEW_SUMMARY.md`
+- Experiment plan: `refine-logs/EXPERIMENT_PLAN.md`
+- Experiment tracker: `refine-logs/EXPERIMENT_TRACKER.md`
+
+## Contribution Snapshot
+- Dominant contribution:
+- Optional supporting contribution:
+- Explicitly rejected complexity:
+
+## Must-Prove Claims
+- [Claim 1]
+- [Claim 2]
+
+## First Runs to Launch
+1. [Run]
+2. [Run]
+3. [Run]
+
+## Main Risks
+- [Risk]:
+- [Mitigation]:
+
+## Next Action
+- Proceed to `/run-experiment`
+```
+
+### Phase 5: Present a Brief Summary to the User
+
+```
+Pipeline complete.
+
+Method output:
+- refine-logs/FINAL_PROPOSAL.md
+
+Experiment output:
+- refine-logs/EXPERIMENT_PLAN.md
+- refine-logs/EXPERIMENT_TRACKER.md
+
+Pipeline summary:
+- refine-logs/PIPELINE_SUMMARY.md
+
+Best next step:
+- /run-experiment
+```
+
+## Output Protocols
+
+> Follow these shared protocols for all output files:
+> - **[Output Versioning Protocol](../../shared-references/output-versioning.md)** — write timestamped file first, then copy to fixed name
+> - **[Output Manifest Protocol](../../shared-references/output-manifest.md)** — log every output to MANIFEST.md
+> - **[Output Language Protocol](../../shared-references/output-language.md)** — respect the project's language setting
+
+## Key Rules
+
+- **Large file handling**: If the Write tool fails due to file size, immediately retry using Bash (`cat << 'EOF' > file`) to write in chunks. Do NOT ask the user for permission — just do it silently.
+
+- Do not let the experiment plan override the Problem Anchor.
+- Do not widen the paper story after method refinement unless a missing validation block is truly necessary.
+- Reuse the same claims across `FINAL_PROPOSAL.md`, `EXPERIMENT_PLAN.md`, and `PIPELINE_SUMMARY.md`.
+- Keep the main paper story compact.
+- If the method is intentionally simple, defend that simplicity in the experiment plan rather than adding new components.
+- If the method uses a modern LLM / VLM / Diffusion / RL primitive, make its necessity test explicit.
+- If the method does not need a frontier primitive, say that clearly and avoid forcing one.
+- Prefer the staged skills when the user only needs one stage; use this skill for the integrated flow.
+
+## Composing with Other Skills
+
+```
+/research-refine-pipeline -> one-shot method + experiment planning
+/research-refine   -> method refinement only
+/experiment-plan   -> experiment planning only
+/run-experiment    -> execution
+```
+

package/skills/skills-codex/research-review/SKILL.md

@@ -0,0 +1,135 @@
+---
+name: "research-review"
+description: "Get a deep critical review of research from GPT using a secondary Codex agent. Use when user says \"review my research\", \"help me review\", \"get external review\", or wants critical feedback on research ideas, papers, or experimental results."
+---
+
+# Research Review via a secondary Codex agent (ultra reasoning)
+
+> **Codex assurance:** the fresh base reviewer is same-family. Record
+> `review_independence: same-family` and `acceptance_status: provisional` in
+> traces and deliverables. A Claude/Gemini overlay may record cross-family
+> accepted; an unavailable reviewer is BLOCKED, never a fabricated PASS.
+
+Get a multi-round critical review of research work from an external LLM with maximum reasoning depth.
+
+## Constants
+
+- REVIEWER_MODEL = `gpt-5.6-sol` — Model used via a secondary Codex agent, reasoning effort `ultra` (deep-audit tier). Must be an OpenAI model (e.g., `gpt-5.6-sol`, `gpt-5.5`, `o3`)
+- **REVIEWER_BACKEND = `codex`** — Default: Codex ultra reviewer (deep-audit tier). Use `--reviewer: oracle-pro` only when explicitly requested; if Oracle is unavailable, warn and fall back to Codex at this skill's declared tier (`ultra`). **Same-family note:** this default reviewer is a second Codex/GPT agent — valid for Type-A completeness/drive review, but not a cross-family Type-B verdict; install a `skills-codex-claude-review` / `skills-codex-gemini-review` overlay for a cross-family acquittal (see `shared-references/reviewer-routing.md`).
+
+## Context: $ARGUMENTS
+
+## Prerequisites
+
+- Use `spawn_agent` and `send_input` when the user has explicitly allowed delegation or subagents.
+- If delegation is not allowed, run the same review loop locally and preserve the same deliverable structure.
+
+## Workflow
+
+### Step 1: Gather Research Context
+Before calling the external reviewer, compile a comprehensive briefing:
+1. Read project narrative documents (e.g., STORY.md, README.md, paper drafts)
+2. Read any memory/notes files for key findings and experiment history
+3. Identify: core claims, methodology, key results, known weaknesses
+
+### Step 2: Initial Review (Round 1)
+Send a detailed prompt with ultra reasoning:
+
+```
+spawn_agent:
+  model: gpt-5.6-sol
+  reasoning_effort: ultra
+  message: |
+    [Full research context + specific questions]
+    Please act as a senior ML reviewer (NeurIPS/ICML level). Start from the
+    assumption that the work is broken somewhere — your job is to find where.
+    Be adversarial. Trust nothing the author tells you — verify everything
+    yourself. Identify:
+    1. Logical gaps or unjustified claims
+    2. Missing experiments that would strengthen the story
+    3. Narrative weaknesses
+    4. Whether the contribution is sufficient for a top venue
+    Please be brutally honest.
+```
+
+### Step 3: Iterative Dialogue (Rounds 2-N)
+Use `send_input` with the returned agent id to continue the conversation:
+
+```text
+send_input:
+  target: [saved reviewer id from Step 2]
+  message: |
+    Please continue the review using the revised materials below.
+
+    Revised files:
+    - /absolute/path/to/file1
+    - /absolute/path/to/file2
+
+    Focus on unresolved weaknesses and whether the revision actually fixed them.
+```
+
+For each round:
+1. **Respond** to criticisms with evidence/counterarguments
+2. **Ask targeted follow-ups** on the most actionable points
+3. **Request specific deliverables**: experiment designs, paper outlines, claims matrices
+
+Key follow-up patterns:
+- "If we reframe X as Y, does that change your assessment?"
+- "What's the minimum experiment to satisfy concern Z?"
+- "Please design the minimal additional experiment package (highest acceptance lift per GPU week)"
+- "Please write a mock NeurIPS/ICML review with scores"
+- "Give me a results-to-claims matrix for possible experimental outcomes"
+
+### Step 4: Convergence
+Stop iterating when:
+- Both sides agree on the core claims and their evidence requirements
+- A concrete experiment plan is established
+- The narrative structure is settled
+
+### Step 5: Document Everything
+Save the full interaction and conclusions to a review document in the project root:
+- Round-by-round summary of criticisms and responses
+- Final consensus on claims, narrative, and experiments
+- Claims matrix (what claims are allowed under each possible outcome)
+- Prioritized TODO list with estimated compute costs
+- Paper outline if discussed
+
+Update project memory/notes with key review conclusions.
+
+If `— composed: <canonical-report-path>` is explicitly present, fold consensus,
+claims matrix, TODOs, and trace links into that report instead of writing a
+standalone review document. Without the directive, write the standalone review
+as documented; never infer composed mode from an existing file. `— standalone`
+always wins. See
+[`output-composition.md`](../shared-references/output-composition.md).
+
+### Step 6: Review Tracing
+
+Save a trace for every `spawn_agent`, `send_input`, or `oracle-pro` review call following `../shared-references/review-tracing.md`. Record the reviewer route, saved agent id, prompt summary, raw response path, decisions, and action items. This preserves the Claude mainline Review Tracing semantics while using Codex-native reviewer calls.
+
+## Key Rules
+
+- ALWAYS use `model: gpt-5.6-sol` + `reasoning_effort: ultra` for reviews (deep-audit tier; capability fallback per `reviewer-routing.md`, never below `xhigh`)
+- Send comprehensive context in Round 1 — the external model cannot read your files
+- Be honest about weaknesses — hiding them leads to worse feedback
+- Push back on criticisms you disagree with, but accept valid ones
+- Focus on ACTIONABLE feedback — "what experiment would fix this?"
+- Document the agent id for potential future resumption
+- The review document should be self-contained (readable without the conversation)
+
+## Prompt Templates
+
+### For initial review:
+"I'm going to present a complete ML research project for your critical review. Please act as a senior ML reviewer (NeurIPS/ICML level)..."
+
+### For experiment design:
+"Please design the minimal additional experiment package that gives the highest acceptance lift per GPU week. Our compute: [describe]. Be very specific about configurations."
+
+### For paper structure:
+"Please turn this into a concrete paper outline with section-by-section claims and figure plan."
+
+### For claims matrix:
+"Please give me a results-to-claims matrix: what claim is allowed under each possible outcome of experiments X and Y?"
+
+### For mock review:
+"Please write a mock NeurIPS review with: Summary, Strengths, Weaknesses, Questions for Authors, Score, Confidence, and What Would Move Toward Accept."