@bohuyeshan/openagent-labforge-core 3.11.1 → 3.11.3

package/generated/skills-bundles/paper/skills/research/research-ideation/auto-claude__idea-discovery-robot/SKILL.md

@@ -0,0 +1,351 @@
+---
+name: "auto-claude/idea-discovery-robot"
+description: "Workflow 1 adaptation for robotics and embodied AI. Orchestrates robotics-aware literature survey, idea generation, novelty check, and internal critique to go from a broad robotics direction to benchmark-grounded, simulation-first ideas. Use when user says \"robotics idea discovery\", \"机器人找idea\", \"embodied AI idea\", \"机器人方向探索\", \"sim2real 选题\", or wants ideas for manipulation, locomotion, navigation, drones, humanoids, or general robot learning."
+argument-hint: ["robotics-direction"]
+allowed-tools: "Bash(*), Read, Write, Edit, Grep, Glob, WebSearch, WebFetch, Agent, Skill, mcp__codex__codex, mcp__codex__codex-reply"
+metadata:
+  category: "research/research-ideation"
+---
+
+# Robotics Idea Discovery Pipeline
+
+Orchestrate a robotics-specific idea discovery workflow for: **$ARGUMENTS**
+
+## Overview
+
+This skill chains three sub-skills into a single automated pipeline:
+
+```
+/research-lit → /idea-creator (robotics framing) → /novelty-check
+  (survey)              (filter + pilot plan)         (verify novel)
+```
+
+But every phase must be grounded in robotics-specific constraints:
+- **Embodiment**: arm, mobile manipulator, drone, humanoid, quadruped, autonomous car, etc.
+- **Task family**: grasping, insertion, locomotion, navigation, manipulation, rearrangement, multi-step planning
+- **Observation + action interface**: RGB/RGB-D/tactile/language; torque/velocity/waypoints/end-effector actions
+- **Simulator / benchmark availability**: simulation-first by default
+- **Real robot constraints**: hardware availability, reset cost, safety, operator time
+- **Evaluation quality**: success rate plus failure cases, safety violations, intervention count, latency, sample efficiency
+- **Sim2real story**: whether the idea can stay in sim, needs offline logs, or truly requires hardware
+
+The goal is not to produce flashy demos. The goal is to produce ideas that are:
+- benchmarkable
+- falsifiable
+- feasible with available robotics infrastructure
+- interesting even if the answer is negative
+
+## Constants
+
+- **MAX_PILOT_IDEAS = 3** — Validate at most 3 top ideas deeply
+- **PILOT_MODE = `sim-first`** — Prefer simulation or offline-log pilots before any hardware execution
+- **REAL_ROBOT_PILOTS = `explicit approval only`** — Never assume physical robot access or approval
+- **AUTO_PROCEED = true** — If user does not respond at checkpoints, proceed with the best sim-first option
+- **TARGET_VENUES = CoRL, RSS, ICRA, IROS, RA-L** — Default novelty and reviewer framing
+
+> Override inline, e.g. `/idea-discovery-robot "bimanual manipulation" — only sim ideas, no real robot` or `/idea-discovery-robot "drone navigation" — focus on CoRL/RSS, 2 pilot ideas max`
+
+## Execution Rule
+
+Follow the phases in order. Do **not** stop after a checkpoint unless:
+- the user explicitly says to stop, or
+- the user asks to change scope and re-run an earlier phase
+
+If `AUTO_PROCEED=true` and the user does not respond, continue immediately to the next phase using the strongest **sim-first, benchmark-grounded** option.
+
+## Phase 0: Frame the Robotics Problem
+
+Before generating ideas, extract or infer this **Robotics Problem Frame** from `$ARGUMENTS` and local project context:
+
+- **Embodiment**
+- **Task family**
+- **Environment type**: tabletop, warehouse, home, outdoor, aerial, driving, legged terrain
+- **Observation modalities**
+- **Action interface / controller abstraction**
+- **Learning regime**: RL, imitation, behavior cloning, world model, planning, VLA/VLM, classical robotics, hybrid
+- **Available assets**: simulator, benchmark suite, teleop data, offline logs, existing codebase, real hardware
+- **Compute budget**
+- **Safety constraints**
+- **Desired contribution type**: method, benchmark, diagnosis, systems, sim2real, data curation
+
+If some fields are missing, make explicit assumptions and default to:
+- **simulation-first**
+- **public benchmark preferred**
+- **no real robot execution**
+
+Write this frame into working notes before moving on. Every later decision should reference it.
+
+## Phase 1: Robotics Literature Survey
+
+Invoke:
+
+```
+/research-lit "$ARGUMENTS — focus venues: CoRL, RSS, ICRA, IROS, RA-L, TRO, Science Robotics"
+```
+
+Then reorganize the findings using a robotics lens instead of a generic ML lens.
+
+### Build a Robotics Landscape Matrix
+
+For each relevant paper, classify:
+
+| Axis | Examples |
+|------|----------|
+| Embodiment | single-arm, mobile manipulator, humanoid, drone, quadruped |
+| Task | pick-place, insertion, navigation, locomotion, long-horizon rearrangement |
+| Learning setup | RL, BC, IL, offline RL, world model, planning, diffusion policy |
+| Observation | RGB, RGB-D, proprioception, tactile, language |
+| Action abstraction | torque, joint velocity, end-effector delta pose, waypoint planner |
+| Eval regime | pure sim, sim+real, real-only, offline benchmark |
+| Benchmark | ManiSkill, RLBench, Isaac Lab, Habitat, Meta-World, CALVIN, LIBERO, custom |
+| Metrics | success rate, collision rate, intervention count, path length, latency, energy |
+| Main bottleneck | sample inefficiency, brittleness, reset cost, perception drift, sim2real gap |
+
+### Search Priorities
+
+When refining the survey, prioritize:
+- recent work from **CoRL, RSS, ICRA, IROS, RA-L**
+- recent arXiv papers from the last 6-12 months
+- benchmark papers and follow-up reproductions
+- negative-result or diagnosis papers if they reveal system bottlenecks
+
+### What to Look For
+
+Do not stop at "who got the best success rate." Explicitly identify:
+- recurring failure modes papers do not fix
+- benchmarks that are saturated or misleading
+- places where embodiment changes invalidate prior conclusions
+- methods that only work with privileged observations
+- ideas whose reported gains come from reset engineering, reward shaping, or hidden infrastructure
+- task families where evaluation quality is weak even if performance numbers look high
+
+**Checkpoint:** Present the landscape to the user in robotics terms:
+
+```
+🤖 Robotics survey complete. I grouped the field by embodiment, benchmark, action interface, and sim2real setup.
+
+Main gaps:
+1. [...]
+2. [...]
+3. [...]
+
+Should I generate ideas under this framing, or should I narrow to a specific robot / benchmark / modality?
+```
+
+- **User approves** (or no response + AUTO_PROCEED=true) → proceed to Phase 2 with the best robotics frame.
+- **User requests changes** (e.g. narrower embodiment, different benchmark family, no sim2real, no hardware) → refine the robotics frame, re-run Phase 1, and present again.
+
+## Phase 2: Robotics-Specific Idea Generation and Filtering
+
+Generate ideas only after the robotics frame is explicit.
+
+Invoke the existing idea generator, but pass the **Robotics Problem Frame** and landscape matrix into the prompt so it does not produce generic ML ideas:
+
+```
+/idea-creator "$ARGUMENTS — robotics frame: [paste Robotics Problem Frame] — focus venues: CoRL, RSS, ICRA, IROS, RA-L — benchmark-specific ideas only — sim-first pilots — no real-robot execution without explicit approval — require failure metrics and baseline clarity"
+```
+
+Then rewrite and filter the output using the robotics-specific rules below.
+
+Each candidate idea must include:
+- **One-sentence summary**
+- **Target embodiment**
+- **Target benchmark / simulator / dataset**
+- **Core bottleneck being addressed**
+- **Minimum sim-first pilot**
+- **Mandatory metrics**
+- **Expected failure mode if the idea does not work**
+- **Whether the idea truly needs real hardware**
+
+### Good Robotics Idea Patterns
+
+Prefer ideas that:
+- expose a real bottleneck in perception-action coupling
+- improve robustness under embodiment or environment shift
+- reduce operator time, reset cost, or demonstration cost
+- strengthen sim2real transfer with measurable mechanisms
+- improve recovery, retry behavior, or failure detection
+- create a better benchmark, diagnostic, or evaluation protocol
+- test an assumption the community repeats but rarely measures
+
+### Weak Robotics Idea Patterns
+
+Downrank ideas that are mostly:
+- "apply a foundation model / VLM / diffusion model to robot X" with no new bottleneck analysis
+- demo-driven but not benchmarkable
+- dependent on inaccessible hardware, custom sensors, or massive private datasets
+- impossible to evaluate without a months-long infrastructure build
+- only interesting if everything works perfectly
+
+### Filtering Rules
+
+For each idea, reject or heavily downrank if:
+- no concrete simulator or benchmark is available
+- no credible baseline exists
+- no measurable metric beyond "looks better"
+- real robot execution is required but hardware access is unclear
+- the setup depends on privileged observations that make the claim weak
+- the expected contribution disappears if evaluation is made fair
+
+**Checkpoint:** Present the ranked robotics ideas before novelty checking:
+
+```
+💡 Robotics ideas generated. Top candidates:
+
+1. [Idea 1] — Embodiment: [...] — Benchmark: [...] — Pilot: sim/offline — Risk: LOW/MEDIUM/HIGH
+2. [Idea 2] — Embodiment: [...] — Benchmark: [...] — Pilot: sim/offline — Risk: LOW/MEDIUM/HIGH
+3. [Idea 3] — requires hardware / weak benchmark / high risk
+
+Should I carry the top sim-first ideas into novelty checking and internal review?
+(If no response, I'll continue with the strongest benchmark-grounded ideas.)
+```
+
+- **User picks ideas** (or no response + AUTO_PROCEED=true) → proceed to Phase 3 with the top sim-first ideas, then continue to Phase 4 and Phase 5.
+- **User wants different constraints** → update the robotics frame and re-run Phase 2.
+- **User wants narrower scope** → go back to Phase 1 with a tighter embodiment / task / benchmark focus.
+
+## Phase 3: Feasibility and Pilot Design
+
+For the top ideas, design a **minimal validation package**.
+
+If the repository already contains a usable simulator, benchmark harness, or offline dataset pipeline, you may validate the top 1-3 ideas there. If not, do **not** force execution. Produce a concrete pilot plan instead.
+
+By default, pilots should be one of:
+- **simulation pilot**
+- **offline log / dataset pilot**
+- **analysis-only pilot** using existing benchmark outputs
+
+Only propose a real-robot pilot if the user explicitly wants that.
+
+For each surviving idea, specify:
+
+```markdown
+- Embodiment:
+- Benchmark / simulator:
+- Baselines:
+- Pilot type: sim / offline / real
+- Compute estimate:
+- Human/operator time:
+- Success metrics:
+- Failure metrics:
+- Safety concerns:
+- What result would count as positive signal:
+- What negative result would still be publishable:
+```
+
+### Real Robot Rule
+
+**Never auto-proceed to physical robot testing.** If an idea needs hardware:
+- mark it as `needs physical validation`
+- design the sim or offline precursor first
+- ask for explicit user confirmation before any real-robot step
+
+If no cheap sim/offline pilot exists, keep the idea in the report but label it **high execution risk**.
+
+After Phase 3, continue to Phase 4 even if you only produced a pilot plan rather than running a pilot. Lack of immediate execution is not a reason to stop the workflow.
+
+## Phase 4: Deep Novelty Verification
+
+For each top idea, run:
+
+```
+/novelty-check "[idea description with embodiment + task family + benchmark + sensor stack + controller/policy class + sim2real angle + target venues: CoRL/RSS/ICRA/IROS/RA-L]"
+```
+
+Robotics novelty checks must include:
+- embodiment
+- task family
+- benchmark / simulator
+- sensor stack
+- controller / policy type
+- sim2real or safety angle if relevant
+
+Be especially skeptical of ideas that are just:
+- old method + new benchmark
+- VLA/VLM + standard manipulation benchmark
+- sim2real claim without new transfer mechanism
+
+If the method is not novel but the **finding** or **evaluation protocol** is, say that explicitly.
+
+## Phase 5: Internal Robotics Review
+
+Perform an internal senior-reviewer pass. Focus on:
+- whether the contribution is really new for robotics, not just ML
+- the minimum benchmark package needed for credibility
+- whether the sim2real story is justified
+- missing baselines or failure analyses
+- whether the idea survives realistic infrastructure constraints
+
+Update the report with the minimum viable evidence package.
+
+## Phase 6: Final Report
+
+Write or update `IDEA_REPORT.md` with a robotics-specific structure so it stays compatible with downstream workflows.
+
+```markdown
+# Robotics Idea Discovery Report
+
+**Direction**: $ARGUMENTS
+**Date**: [today]
+**Pipeline**: research-lit → idea-creator (robotics framing) → novelty-check → internal review
+
+## Robotics Problem Frame
+- Embodiment:
+- Task family:
+- Observation / action interface:
+- Available assets:
+- Constraints:
+
+## Landscape Matrix
+[grouped by embodiment, benchmark, and bottleneck]
+
+## Ranked Ideas
+
+### Idea 1: [title] — RECOMMENDED
+- Embodiment:
+- Benchmark / simulator:
+- Bottleneck addressed:
+- Pilot type: sim / offline / real
+- Positive signal:
+- Novelty:
+- Internal review notes:
+- Hardware risk:
+- Next step:
+
+## Eliminated Ideas
+- [idea] — killed because benchmark unclear / hardware inaccessible / novelty weak / no fair evaluation
+
+## Evidence Package for the Top Idea
+- Required baselines:
+- Required metrics:
+- Required failure cases:
+- Whether real robot evidence is mandatory:
+
+## Next Steps
+- [ ] Implement sim-first pilot
+- [ ] Run /novelty-check on the final idea wording
+- [ ] Only after approval: consider hardware validation
+```
+
+## Key Rules
+
+- **Simulation first.** Hardware is never the default.
+- **Benchmark specificity is mandatory.** No benchmark, no serious idea.
+- **Evaluation must include failures.** Success rate alone is not enough.
+- **Embodiment matters.** Do not assume a result on one robot transfers to another.
+- **Avoid foundation-model theater.** Novel terminology is not novelty.
+- **Infrastructure realism matters.** Operator time, reset burden, and safety count as research constraints.
+- **If the contribution is mainly diagnostic or evaluative, say so.** That can still be publishable.
+
+## Composing with Later Work
+
+After this workflow identifies a strong robotics idea:
+
+```
+/idea-discovery-robot "direction"   ← you are here
+implement sim-first pilot
+/run-experiment                     ← if infrastructure exists
+/ulw-loop "top robotics idea"
+```
+
+If no simulator or benchmark is available yet, stop at the report and ask the user to choose whether to build infrastructure or pivot to a more executable idea.

package/generated/skills-bundles/paper/skills/research/research-ideation/auto-claude__research-pipeline/SKILL.md

@@ -0,0 +1,174 @@
+---
+name: "auto-claude/research-pipeline"
+description: "Full research pipeline: Workflow 1 (idea discovery) → implementation → Workflow 2 (ULTRAWORK loop). Goes from a broad research direction all the way to a submission-ready paper. Use when user says \"全流程\", \"full pipeline\", \"从找idea到投稿\", \"end-to-end research\", or wants the complete autonomous research lifecycle."
+argument-hint: ["research-direction"]
+allowed-tools: "Bash(*), Read, Write, Edit, Grep, Glob, WebSearch, WebFetch, Agent, Skill, mcp__codex__codex, mcp__codex__codex-reply"
+metadata:
+  category: "research/research-ideation"
+---
+
+# Full Research Pipeline: Idea → Experiments → Submission
+
+End-to-end autonomous research workflow for: **$ARGUMENTS**
+
+## Constants
+
+- **AUTO_PROCEED = true** — When `true`, Gate 1 auto-selects the top-ranked idea (highest pilot signal + novelty confirmed) and continues to implementation. When `false`, always waits for explicit user confirmation before proceeding.
+- **ARXIV_DOWNLOAD = false** — When `true`, `/research-lit` downloads the top relevant arXiv PDFs during literature survey. When `false` (default), only fetches metadata via arXiv API. Passed through to `/idea-discovery` → `/research-lit`.
+- **HUMAN_CHECKPOINT = false** — When `true`, the ULTRAWORK loop (Stage 4) pauses after each round to let you inspect progress and provide custom modification instructions before fixes are implemented. When `false` (default), the loop runs fully autonomously. Passed through to `/ulw-loop`.
+
+> 💡 Override via argument, e.g., `/research-pipeline "topic" — AUTO_PROCEED: false, human checkpoint: true`.
+
+## Overview
+
+This skill chains the entire research lifecycle into a single pipeline:
+
+```
+/idea-discovery → implement → /run-experiment → /ulw-loop → submission-ready
+├── Workflow 1 ──┤            ├────────── Workflow 2 ──────────────┤
+```
+
+It orchestrates two major workflows plus the implementation bridge between them.
+
+## Pipeline
+
+### Stage 1: Idea Discovery (Workflow 1)
+
+Invoke the idea discovery pipeline:
+
+```
+/idea-discovery "$ARGUMENTS"
+```
+
+This internally runs: `/research-lit` → `/idea-creator` → `/novelty-check`
+
+**Output:** `IDEA_REPORT.md` with ranked, validated, pilot-tested ideas.
+
+**🚦 Gate 1 — Human Checkpoint:**
+
+After `IDEA_REPORT.md` is generated, **pause and present the top ideas to the user**:
+
+```
+📋 Idea Discovery complete. Top ideas:
+
+1. [Idea 1 title] — Pilot: POSITIVE (+X%), Novelty: CONFIRMED
+2. [Idea 2 title] — Pilot: WEAK POSITIVE (+Y%), Novelty: CONFIRMED
+3. [Idea 3 title] — Pilot: NEGATIVE, eliminated
+
+Recommended: Idea 1. Shall I proceed with implementation?
+```
+
+**If AUTO_PROCEED=false:** Wait for user confirmation before continuing. The user may:
+- **Approve an idea** → proceed to Stage 2.
+- **Pick a different idea** → proceed with their choice.
+- **Request changes** (e.g., "combine Idea 1 and 3", "focus more on X") → update the idea prompt with user feedback, re-run `/idea-discovery` with refined constraints, and present again.
+- **Reject all ideas** → collect feedback on what's missing, re-run Stage 1 with adjusted research direction. Repeat until the user commits to an idea.
+- **Stop here** → save current state to `IDEA_REPORT.md` for future reference.
+
+**If AUTO_PROCEED=true:** Present the top ideas, wait 10 seconds for user input. If no response, auto-select the #1 ranked idea (highest pilot signal + novelty confirmed) and proceed to Stage 2. Log: `"AUTO_PROCEED: selected Idea 1 — [title]"`.
+
+> ⚠️ **This gate waits for user confirmation when AUTO_PROCEED=false.** When `true`, it auto-selects the top idea after presenting results. The rest of the pipeline (Stages 2-4) is expensive (GPU time + multiple review rounds), so set `AUTO_PROCEED=false` if you want to manually choose which idea to pursue.
+
+### Stage 2: Implementation
+
+Once the user confirms which idea to pursue:
+
+1. **Read the idea details** from `IDEA_REPORT.md` (hypothesis, experimental design, pilot code)
+
+2. **Implement the full experiment**:
+   - Extend pilot code to full scale (multi-seed, full dataset, proper baselines)
+   - Add proper evaluation metrics and logging (wandb if configured)
+   - Write clean, reproducible experiment scripts
+   - Follow existing codebase conventions
+
+3. **Code review**: Before deploying, do a self-review:
+   - Are all hyperparameters configurable via argparse?
+   - Is the random seed fixed and controllable?
+   - Are results saved to JSON/CSV for later analysis?
+   - Is there proper logging for debugging?
+
+### Stage 3: Deploy Experiments (Workflow 2 — Part 1)
+
+Deploy the full-scale experiments:
+
+```
+/run-experiment [experiment command]
+```
+
+**What this does:**
+- Check GPU availability on configured servers
+- Sync code to remote server
+- Launch experiments in screen sessions with proper CUDA_VISIBLE_DEVICES
+- Verify experiments started successfully
+
+**Monitor progress:**
+
+```
+/monitor-experiment [server]
+```
+
+Wait for experiments to complete. Collect results.
+
+### Stage 4: ULTRAWORK Loop (Workflow 2 — Part 2)
+
+Once initial results are in, start the autonomous improvement loop:
+
+```
+/ulw-loop "$ARGUMENTS — [chosen idea title]"
+```
+
+**What this does (recommended max 4 rounds):**
+1. Run work → check progress → record evidence
+2. Oracle verifies completion; if not verified, continue iterating
+3. Deploy fixes, collect new results
+4. Repeat until Oracle verification or max rounds reached
+
+**Output:** Maintain a `ULTRAWORK_LOG.md` with each round's changes and evidence.
+
+### Stage 5: Final Summary
+
+After the ULTRAWORK loop completes, write a final status report:
+
+```markdown
+# Research Pipeline Report
+
+**Direction**: $ARGUMENTS
+**Chosen Idea**: [title]
+**Date**: [start] → [end]
+**Pipeline**: idea-discovery → implement → run-experiment → ulw-loop
+
+## Journey Summary
+- Ideas generated: X → filtered to Y → piloted Z → chose 1
+- Implementation: [brief description of what was built]
+- Experiments: [number of GPU experiments, total compute time]
+- Review rounds: N/4, final score: X/10
+
+## Final Status
+- [ ] Ready for submission / [ ] Needs manual follow-up
+
+## Remaining TODOs (if any)
+- [items flagged by reviewer that weren't addressed]
+
+## Files Changed
+- [list of key files created/modified]
+```
+
+## Key Rules
+
+- **Human checkpoint after Stage 1 is controlled by AUTO_PROCEED.** When `false`, do not proceed without user confirmation. When `true`, auto-select the top idea after presenting results.
+- **Stages 2-4 can run autonomously** once the user confirms the idea. This is the "sleep and wake up to results" part.
+- **If Stage 4 hits the max rounds without Oracle verification**, stop and report remaining issues. Do not loop forever.
+- **Budget awareness**: Track total GPU-hours across the pipeline. Flag if approaching user-defined limits.
+- **Documentation**: Every stage updates its own output file. The full history should be self-contained.
+- **Fail gracefully**: If any stage fails (no good ideas, experiments crash, review loop stuck), report clearly and suggest alternatives rather than forcing forward.
+
+## Typical Timeline
+
+| Stage | Duration | Can sleep? |
+|-------|----------|------------|
+| 1. Idea Discovery | 30-60 min | Yes if AUTO_PROCEED=true |
+| 2. Implementation | 15-60 min | Yes (autonomous after Gate 1) |
+| 3. Deploy | 5 min + experiment time | Yes ✅ |
+| 4. ULTRAWORK | 1-4 hours (depends on experiments) | Yes ✅ |
+
+**Sweet spot**: Run Stage 1-2 in the evening, launch Stage 3-4 before bed, wake up to a reviewed paper.