@uzysjung/agent-harness 26.151.0 → 26.153.0

package/templates/skills/deep-research/SKILL.md

@@ -1,179 +0,0 @@
----
-name: deep-research
-description: Multi-source deep research using firecrawl and exa MCPs. Searches the web, synthesizes findings, and delivers cited reports with source attribution. Use when the user wants thorough research on any topic with evidence and citations.
-origin: ECC
----
-
-# Deep Research
-
-Produce thorough, cited research reports from multiple web sources using firecrawl and exa MCP tools.
-
-## When to Activate
-
-- User asks to research any topic in depth
-- Competitive analysis, technology evaluation, or market sizing
-- Due diligence on companies, investors, or technologies
-- Any question requiring synthesis from multiple sources
-- User says "research", "deep dive", "investigate", or "what's the current state of"
-
-## Web access
-
-Use the firecrawl / exa MCP tools when they are configured — they give the best recall. **The
-harness does not install either**, so check what you actually have first: without them, run the same
-workflow on your CLI's built-in web search and fetch tools (in Claude Code, `WebSearch` /
-`WebFetch`). With no web access at all, say so instead of writing a report from memory — an uncited
-report is the failure mode this skill exists to prevent.
-
-## Workflow
-
-### Step 1: Understand the Goal
-
-Ask 1-2 quick clarifying questions:
-- "What's your goal — learning, making a decision, or writing something?"
-- "Any specific angle or depth you want?"
-
-If the user says "just research it" — skip ahead with reasonable defaults.
-
-### Step 2: Plan the Research
-
-Break the topic into 3-5 research sub-questions. Example:
-- Topic: "Impact of AI on healthcare"
-  - What are the main AI applications in healthcare today?
-  - What clinical outcomes have been measured?
-  - What are the regulatory challenges?
-  - What companies are leading this space?
-  - What's the market size and growth trajectory?
-
-### Step 3: Execute Multi-Source Search
-
-For EACH sub-question, search using available MCP tools:
-
-**With firecrawl:**
-```
-firecrawl_search(query: "<sub-question keywords>", limit: 8)
-```
-
-**With exa:**
-```
-web_search_exa(query: "<sub-question keywords>", numResults: 8)
-web_search_advanced_exa(query: "<keywords>", numResults: 5, startPublishedDate: "2025-01-01")
-```
-
-**Search strategy:**
-- Use 2-3 different keyword variations per sub-question
-- Mix general and news-focused queries
-- Aim for 15-30 unique sources total
-- Prioritize: academic, official, reputable news > blogs > forums
-
-### Step 4: Deep-Read Key Sources
-
-For the most promising URLs, fetch full content:
-
-**With firecrawl:**
-```
-firecrawl_scrape(url: "<url>")
-```
-
-**With exa:**
-```
-crawling_exa(url: "<url>", tokensNum: 5000)
-```
-
-Read 3-5 key sources in full for depth. Do not rely only on search snippets.
-
-### Step 5: Synthesize and Write Report
-
-Structure the report:
-
-```markdown
-# [Topic]: Research Report
-*Generated: [date] | Sources: [N] | Confidence: [High/Medium/Low]*
-
-## Executive Summary
-[3-5 sentence overview of key findings]
-
-## 1. [First Major Theme]
-[Findings with inline citations]
-- Key point ([Source Name](url))
-- Supporting data ([Source Name](url))
-
-## 2. [Second Major Theme]
-...
-
-## 3. [Third Major Theme]
-...
-
-## Key Takeaways
-- [Actionable insight 1]
-- [Actionable insight 2]
-- [Actionable insight 3]
-
-## Sources
-1. [Title](url) — [one-line summary]
-2. ...
-
-## Research Ledger — [N] confirmed · [M] killed
-
-| Killed claim/direction | Why rejected | Source that killed it |
-|---|---|---|
-| [what you looked into] | [contradicting evidence / legal or licensing block / vendor-only claim] | [url] |
-
-> **Caveats**: [which numbers are vendor-promotional or self-reported, which sources are
-> preprints, which claims you could not verify verbatim and are therefore follow-up work]
-
-## Methodology
-Searched [N] queries across web and news. Analyzed [M] sources.
-Sub-questions investigated: [list]
-```
-
-### The ledger is not optional
-
-A report that lists only what survived hides the most reusable part of the work. Record it as
-`N confirmed · M killed`, and for each killed item write **why** — contradicting evidence, a
-legal block, or a claim that turned out to be vendor-promotional. Two things follow:
-
-- **Nobody re-researches a dead end.** Next quarter the same idea resurfaces; the ledger answers
-  it in one line instead of another research cycle.
-- **A kill count is evidence of rigor.** `22 confirmed · 0 killed` means you were collecting
-  support, not testing claims. Zero kills is a signal to re-examine, not a clean result.
-
-The caveat block does the same job for what survived but is *weakly* sourced: vendor numbers with
-no independent verification, preprints whose self-reported performance you cite architecture from
-but not results, beta APIs. Naming them keeps a soft source from hardening into a fact downstream.
-
-### Step 6: Deliver
-
-- **Short topics**: Post the full report in chat
-- **Long reports**: Post the executive summary + key takeaways, save full report to a file
-
-## Parallel Research with Subagents
-
-For broad topics, use Claude Code's Task tool to parallelize:
-
-```
-Launch 3 research agents in parallel:
-1. Agent 1: Research sub-questions 1-2
-2. Agent 2: Research sub-questions 3-4
-3. Agent 3: Research sub-question 5 + cross-cutting themes
-```
-
-Each agent searches, reads sources, and returns findings. The main session synthesizes into the final report.
-
-## Quality Rules
-
-1. **Every claim needs a source.** No unsourced assertions.
-2. **Cross-reference.** If only one source says it, flag it as unverified.
-3. **Recency matters.** Prefer sources from the last 12 months.
-4. **Acknowledge gaps.** If you couldn't find good info on a sub-question, say so.
-5. **No hallucination.** If you don't know, say "insufficient data found."
-6. **Separate fact from inference.** Label estimates, projections, and opinions clearly.
-
-## Examples
-
-```
-"Research the current state of nuclear fusion energy"
-"Deep dive into Rust vs Go for backend services in 2026"
-"Research the best strategies for bootstrapping a SaaS business"
-"What's happening with the US housing market right now?"
-"Investigate the competitive landscape for AI code editors"
-```

package/templates/skills/deep-research/agents/openai.yaml

@@ -1,7 +0,0 @@
-interface:
-  display_name: "Deep Research"
-  short_description: "Multi-source deep research with firecrawl and exa MCPs"
-  brand_color: "#6366F1"
-  default_prompt: "Research the given topic using firecrawl and exa, produce a cited report"
-policy:
-  allow_implicit_invocation: true

package/templates/skills/eval-harness/SKILL.md

@@ -1,306 +0,0 @@
----
-name: eval-harness
-description: Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles
-origin: ECC
-tools: Read, Write, Edit, Bash, Grep, Glob
----
-
-# Eval Harness Skill
-
-A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.
-
-## When to Activate
-
-- Setting up eval-driven development (EDD) for AI-assisted workflows
-- Defining pass/fail criteria for Claude Code task completion
-- Measuring agent reliability with pass@k metrics
-- Creating regression test suites for prompt or agent changes
-- Benchmarking agent performance across model versions
-
-## Philosophy
-
-Eval-Driven Development treats evals as the "unit tests of AI development":
-- Define expected behavior BEFORE implementation
-- Run evals continuously during development
-- Track regressions with each change
-- Use pass@k metrics for reliability measurement
-
-## Eval Types
-
-### Capability Evals
-Test if Claude can do something it couldn't before:
-```markdown
-[CAPABILITY EVAL: feature-name]
-Task: Description of what Claude should accomplish
-Success Criteria:
-  - [ ] Criterion 1
-  - [ ] Criterion 2
-  - [ ] Criterion 3
-Expected Output: Description of expected result
-```
-
-### Regression Evals
-Ensure changes don't break existing functionality:
-```markdown
-[REGRESSION EVAL: feature-name]
-Baseline: SHA or checkpoint name
-Tests:
-  - existing-test-1: PASS/FAIL
-  - existing-test-2: PASS/FAIL
-  - existing-test-3: PASS/FAIL
-Result: X/Y passed (previously Y/Y)
-```
-
-## Grader Types
-
-### 1. Code-Based Grader
-Deterministic checks using code:
-```bash
-# Check if file contains expected pattern
-grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
-
-# Check if tests pass
-npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
-
-# Check if build succeeds
-npm run build && echo "PASS" || echo "FAIL"
-```
-
-### 2. Model-Based Grader
-Use Claude to evaluate open-ended outputs:
-```markdown
-[MODEL GRADER PROMPT]
-Evaluate the following code change:
-1. Does it solve the stated problem?
-2. Is it well-structured?
-3. Are edge cases handled?
-4. Is error handling appropriate?
-
-Score: 1-5 (1=poor, 5=excellent)
-Reasoning: [explanation]
-```
-
-### 3. Human Grader
-Flag for manual review:
-```markdown
-[HUMAN REVIEW REQUIRED]
-Change: Description of what changed
-Reason: Why human review is needed
-Risk Level: LOW/MEDIUM/HIGH
-```
-
-## Metrics
-
-### pass@k
-"At least one success in k attempts"
-- pass@1: First attempt success rate
-- pass@3: Success within 3 attempts
-- Typical target: pass@3 > 90%
-
-### pass^k
-"All k trials succeed"
-- Higher bar for reliability
-- pass^3: 3 consecutive successes
-- Use for critical paths
-
-## Eval Workflow
-
-### 1. Define (Before Coding)
-
-Write the spec to a file (`.claude/evals/<feature>.md`) before implementing. Give every eval a
-stable ID — `C1..Cn` for capability, `R1..Rn` for regression — so the same identifier carries
-from definition to the post-implementation status line, and a reviewer can check them off one by
-one. Prose-only lists ("Can create new user account") can't be referenced or scored.
-
-````markdown
-# EVAL: <feature name> (<phase/PR>)
-
-**Feature**: <one line>
-**Baseline**: commit <sha>          # what "regression" is measured against
-**Target**: pass@1 = 100% for capability evals
-
-## Capability Evals
-
-### C1: <name>
-- <concrete, checkable expectation — inputs → expected output>
-
-### C2: <name>
-- <expectation>
-
-## Regression Evals
-
-### R1: <existing behavior that must not move>
-- <expectation>
-
-## Test Command
-```bash
-pytest tests/test_<area>.py -v -k "<selector>"
-```
-
-## Status (after implementation)
-- C1-Cn: PASS via <tests (N개) / route check / manual>
-- R1-Rn: PASS
-
-**Overall: pass@1 = <x>%**
-````
-
-Two fields carry most of the weight. **Baseline commit** makes "regression" falsifiable — without
-it, R-evals are opinions about the past. **Test Command** makes the spec re-runnable by someone
-who didn't write it; an eval nobody can re-run is documentation, not a gate.
-
-Fill the Status section *after* implementing, in the same file. A spec whose status is still empty
-at merge time means the evals were written and never used.
-
-### 2. Implement
-Write code to pass the defined evals.
-
-### 3. Evaluate
-```bash
-# Run capability evals
-[Run each capability eval, record PASS/FAIL]
-
-# Run regression evals
-npm test -- --testPathPattern="existing"
-
-# Generate report
-```
-
-### 4. Report
-```markdown
-EVAL REPORT: feature-xyz
-========================
-
-Capability Evals:
-  create-user:     PASS (pass@1)
-  validate-email:  PASS (pass@2)
-  hash-password:   PASS (pass@1)
-  Overall:         3/3 passed
-
-Regression Evals:
-  login-flow:      PASS
-  session-mgmt:    PASS
-  logout-flow:     PASS
-  Overall:         3/3 passed
-
-Metrics:
-  pass@1: 67% (2/3)
-  pass@3: 100% (3/3)
-
-Status: READY FOR REVIEW
-```
-
-## Integration Patterns
-
-### Pre-Implementation
-```
-/eval define feature-name
-```
-Creates eval definition file at `.claude/evals/feature-name.md`
-
-### During Implementation
-```
-/eval check feature-name
-```
-Runs current evals and reports status
-
-### Post-Implementation
-```
-/eval report feature-name
-```
-Generates full eval report
-
-## Eval Storage (.md + .log Pair Format)
-
-각 평가 항목은 **`<topic>.md` (설계) + `<topic>.log` (실행 결과)** 쌍으로 저장. 강제. 단독 .md만 있으면 재현 불가.
-
-```
-.claude/
-  evals/
-    feature-xyz.md        # Eval definition (Capability/Regression/Test 3섹션 필수)
-    feature-xyz.log       # Eval run history (실행 시각, grader, pass/fail)
-    session-YYYYMMDD.md   # 세션 단위 회고 + 차기 backlog
-    session-YYYYMMDD.log  # 동일 세션의 grader 출력
-    baseline.json         # Regression baselines (선택)
-```
-
-> eval 산출물은 `docs/evals/*.{md,log}` 로 모은다 — 실행 로그와 판정을 같은 자리에 둔다.
-
-### .md 파일 의무 섹션 (3개)
-
-```markdown
-# Eval: <topic>
-
-## Capability
-[새 능력 — Claude/agent가 무엇을 할 수 있는지]
-- AC: [측정 가능 기준]
-- Grader: code-based / model-based / human
-
-## Regression
-[기존 기능 보호 — 변경으로 깨지면 안 되는 baseline]
-- Baseline: <SHA or checkpoint>
-- Tests: [목록]
-
-## Test
-[실행 절차 — 누가 다시 돌려도 동일 결과 나와야 함]
-- Setup: [사전 조건]
-- Run: `bash run-eval.sh <topic>` 또는 명시적 명령
-- Expected: [기대 출력]
-```
-
-### .log 파일 형식
-
-각 실행마다 append. 시간순 누적.
-
-```
-=== 2026-04-19 14:32 (run #1) ===
-Capability: 3/3 PASS (pass@1)
-Regression: 5/5 PASS (pass^3)
-Status: SHIP READY
-
-=== 2026-04-20 09:15 (run #2 — after refactor) ===
-Capability: 3/3 PASS
-Regression: 4/5 PASS (login-flow regressed at SHA abc123)
-Status: BLOCKED — fix login-flow first
-```
-
-## Best Practices
-
-1. **Define evals BEFORE coding** - Forces clear thinking about success criteria
-2. **Run evals frequently** - Catch regressions early
-3. **Track pass@k over time** - Monitor reliability trends
-4. **Use code graders when possible** - Deterministic > probabilistic
-5. **Human review for security** - Never fully automate security checks
-6. **Keep evals fast** - Slow evals don't get run
-7. **Version evals with code** - Evals are first-class artifacts
-
-## Example: Adding Authentication
-
-```markdown
-## EVAL: add-authentication
-
-### Phase 1: Define (10 min)
-Capability Evals:
-- [ ] User can register with email/password
-- [ ] User can login with valid credentials
-- [ ] Invalid credentials rejected with proper error
-- [ ] Sessions persist across page reloads
-- [ ] Logout clears session
-
-Regression Evals:
-- [ ] Public routes still accessible
-- [ ] API responses unchanged
-- [ ] Database schema compatible
-
-### Phase 2: Implement (varies)
-[Write code]
-
-### Phase 3: Evaluate
-Run: /eval check add-authentication
-
-### Phase 4: Report
-EVAL REPORT: add-authentication
-==============================
-Capability: 5/5 passed (pass@3: 100%)
-Regression: 3/3 passed (pass^3: 100%)
-Status: SHIP IT
-```

package/templates/skills/eval-harness/agents/openai.yaml

@@ -1,7 +0,0 @@
-interface:
-  display_name: "Eval Harness"
-  short_description: "Eval-driven development with pass/fail criteria"
-  brand_color: "#EC4899"
-  default_prompt: "Set up eval-driven development with pass/fail criteria"
-policy:
-  allow_implicit_invocation: true