@uzysjung/agent-harness 26.112.0 → 26.114.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@uzysjung/agent-harness",
-  "version": "26.112.0",
+  "version": "26.114.0",
   "description": "Curate vetted AI-coding skills & plugins by your tech stack — install only what you need, across Claude Code, Codex, OpenCode & Antigravity",
   "type": "module",
   "publishConfig": {

package/templates/rules/benchmark-parity.md

@@ -1,7 +1,8 @@
 # Benchmark Parity (벤치마크 핵심 기능 완결성)
 
 레퍼런스 제품(벤치마크)을 실측해 핵심 기능을 도출하고, 자체 구현이 **올바르게 + 필요한 항목 전부**
-갖췄는지 반복 검토·발전시키는 워크플로우. 실서비스 운영(레퍼런스 audit 20회 내외 실행)에서 검증된
+갖췄는지 반복 검토·발전시키는 워크플로우. **기준선은 레퍼런스 또는 자기 배포본**(후자 = Dogfood
+pass) — 비교 대상이 다를 뿐 산출물(gap.md)과 규율은 같다. 실서비스 운영(레퍼런스 audit 20회 내외 실행)에서 검증된
 관행의 일반화. 목표는 **모방이 아니라 핵심 기능 완결성** — NORTH_STAR 핵심 경쟁력에 기여하지 않는
 단순 모방은 deferred (Will/Won't).
 
@@ -48,13 +49,29 @@ SSOT — 없으면 갭 판정 기준 자체가 없다.
 - **증거 = file:line + capture 참조.** 증거 없는 행은 갭이 아니라 추정이다 (기재 금지).
 - 선택 확장 열: Effort(S/M/L) · 벤치마크 구분(내부 버그 vs divergence).
 
+## Dogfood pass (배포본 전수 walkthrough)
+
+벤치마크 비교가 "남과 비교"라면, dogfood 는 **배포된 자기 서비스를 사용자처럼 끝까지 써보는
+것**이다. 둘은 기준선이 다를 뿐 산출물은 같다 — **새 스키마를 만들지 말고 위 gap.md 표를 그대로
+쓴다** (증거 열에 capture/영상 경로).
+
+- **대상 = 배포본**(로컬 dev 아님). 로그인 상태로 전 메뉴를 실제로 눌러본다.
+- **범위를 먼저 적는다** — "전체 서비스 / 주요 메뉴 N개" 처럼. 범위 미기재 보고는 커버리지를
+  부풀린다.
+- **심각도 롤업을 머리에 둔다**: `CRITICAL n · HIGH n · MEDIUM n · LOW n · 합계 n`. 개별 행만
+  나열하면 "얼마나 나쁜가"가 안 보인다.
+- **재현 아티팩트 필수** — 이슈마다 스크린샷/영상 경로. 없으면 증거 없는 행과 같다.
+- **게이트**: CRITICAL 0 이어야 dogfood 통과. HIGH 는 사유를 적고 사용자 판단으로 넘긴다.
+  (아래 "자율 루프 완료조건"의 `CRITICAL/HIGH 전부 [x]` 와 다른 게이트다 — 이쪽은 **패스 통과**
+  기준, 저쪽은 **루프 종료** 기준. 루프를 끝내려면 HIGH 까지 닫아야 한다.)
+
 ## PR 의무 필드 (UI/UX 변경 PR)
 
 UI/UX 에 영향하는 PR 의 description 에 다음 섹션 필수:
 
 ```markdown
 ## Fidelity (benchmark parity)
-- Benchmark: <벤치마크명>
+- Benchmark: <벤치마크명> (dogfood 발 PR 이면 `dogfood — 자기 배포본`)
 - Capture: docs/research/<area>_audit_<sprint>/<file>.png
 - 갭 매트릭스: docs/research/<area>_audit_<sprint>/gap.md §X
 ```

package/templates/skills/deep-research/SKILL.md

@@ -112,11 +112,35 @@ Structure the report:
 1. [Title](url) — [one-line summary]
 2. ...
 
+## Research Ledger — [N] confirmed · [M] killed
+
+| Killed claim/direction | Why rejected | Source that killed it |
+|---|---|---|
+| [what you looked into] | [contradicting evidence / legal or licensing block / vendor-only claim] | [url] |
+
+> **Caveats**: [which numbers are vendor-promotional or self-reported, which sources are
+> preprints, which claims you could not verify verbatim and are therefore follow-up work]
+
 ## Methodology
 Searched [N] queries across web and news. Analyzed [M] sources.
 Sub-questions investigated: [list]
 ```
 
+### The ledger is not optional
+
+A report that lists only what survived hides the most reusable part of the work. Record it as
+`N confirmed · M killed`, and for each killed item write **why** — contradicting evidence, a
+legal block, or a claim that turned out to be vendor-promotional. Two things follow:
+
+- **Nobody re-researches a dead end.** Next quarter the same idea resurfaces; the ledger answers
+  it in one line instead of another research cycle.
+- **A kill count is evidence of rigor.** `22 confirmed · 0 killed` means you were collecting
+  support, not testing claims. Zero kills is a signal to re-examine, not a clean result.
+
+The caveat block does the same job for what survived but is *weakly* sourced: vendor numbers with
+no independent verification, preprints whose self-reported performance you cite architecture from
+but not results, beta APIs. Naming them keeps a soft source from hardening into a fact downstream.
+
 ### Step 6: Deliver
 
 - **Short topics**: Post the full report in chat

package/templates/skills/eval-harness/SKILL.md

@@ -106,24 +106,51 @@ Risk Level: LOW/MEDIUM/HIGH
 ## Eval Workflow
 
 ### 1. Define (Before Coding)
-```markdown
-## EVAL DEFINITION: feature-xyz
 
-### Capability Evals
-1. Can create new user account
-2. Can validate email format
-3. Can hash password securely
+Write the spec to a file (`.claude/evals/<feature>.md`) before implementing. Give every eval a
+stable ID — `C1..Cn` for capability, `R1..Rn` for regression — so the same identifier carries
+from definition to the post-implementation status line, and a reviewer can check them off one by
+one. Prose-only lists ("Can create new user account") can't be referenced or scored.
 
-### Regression Evals
-1. Existing login still works
-2. Session management unchanged
-3. Logout flow intact
+````markdown
+# EVAL: <feature name> (<phase/PR>)
+
+**Feature**: <one line>
+**Baseline**: commit <sha>          # what "regression" is measured against
+**Target**: pass@1 = 100% for capability evals
+
+## Capability Evals
+
+### C1: <name>
+- <concrete, checkable expectation — inputs → expected output>
+
+### C2: <name>
+- <expectation>
 
-### Success Metrics
-- pass@3 > 90% for capability evals
-- pass^3 = 100% for regression evals
+## Regression Evals
+
+### R1: <existing behavior that must not move>
+- <expectation>
+
+## Test Command
+```bash
+pytest tests/test_<area>.py -v -k "<selector>"
 ```
 
+## Status (after implementation)
+- C1-Cn: PASS via <tests (N개) / route check / manual>
+- R1-Rn: PASS
+
+**Overall: pass@1 = <x>%**
+````
+
+Two fields carry most of the weight. **Baseline commit** makes "regression" falsifiable — without
+it, R-evals are opinions about the past. **Test Command** makes the spec re-runnable by someone
+who didn't write it; an eval nobody can re-run is documentation, not a gate.
+
+Fill the Status section *after* implementing, in the same file. A spec whose status is still empty
+at merge time means the evals were written and never used.
+
 ### 2. Implement
 Write code to pass the defined evals.
 

package/templates/skills/model-orchestration/SKILL.md

@@ -164,6 +164,19 @@ it. How that plays out per lane:
 This pairs with, not replaces, deterministic gates (tests, typecheck, CI) — the verifier
 judges what automation can't: spec fit, missed edge cases, design drift.
 
+**Verdict vocabulary (fixed).** A verifier's report never ends in free prose — it ends with
+exactly one verdict, `PASS` / `PASS_WITH_NITS` / `FAIL`, and every finding carries one
+severity label, `CRITICAL` / `HIGH` / `MEDIUM` / `LOW` (full contract: the verification-loop
+skill's Verdict Contract). Two consequences for orchestration:
+
+- **FAIL closes only on re-verification.** Fix → re-verify is one cycle; if the same
+  reviewer will re-verify, keep it alive via SendMessage and *declare* that intent
+  (worker lifecycle) — otherwise spawn a fresh verifier.
+- **PASS_WITH_NITS is not silent PASS.** Its LOW/MEDIUM findings come back to the
+  orchestrator as recorded follow-ups, not as buried prose — deciding their fate is an
+  orchestrator judgment call, never the verifier's. (The name says nits, but the tier it
+  covers is LOW **and** MEDIUM.)
+
 ## Orchestrator handoff (quota exhaustion)
 
 When the top-tier orchestrator's quota runs out mid-project, **Opus @ `max` takes over
@@ -203,6 +216,7 @@ build on. Hand off manually:
                                → 오케스트레이터(Fable) 직접
 기획·스펙·계획 문서 작성·관리 / 핵심 구현 / V&V
                                → opus  @ xhigh (또는 max)   — pinned role 또는 Workflow opts
+V&V 판정                       → PASS | PASS_WITH_NITS | FAIL + CRITICAL~LOW (verification-loop 계약)
 반복 구현 / E2E 테스트 / 리서치 스윕
                                → sonnet @ high 이상
 결정적 변환 (rename·포맷)      → 모델 위임 금지 — sed/grep/스크립트 직접

package/templates/skills/verification-loop/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: verification-loop
-description: "A comprehensive verification system for Claude Code sessions."
+description: "A comprehensive verification system for Claude Code sessions. Every run ends with a fixed verdict — PASS / PASS_WITH_NITS / FAIL — plus severity-labeled findings (CRITICAL/HIGH/MEDIUM/LOW)."
 origin: ECC
 ---
 
@@ -26,7 +26,9 @@ npm run build 2>&1 | tail -20
 pnpm build 2>&1 | tail -20
 ```
 
-If build fails, STOP and fix before continuing.
+If build fails, STOP and fix — then re-verify from Phase 1 in a fresh instance. Continuing
+through the remaining phases yourself would make you the verifier of code you just wrote,
+which the Verdict Contract below forbids.
 
 ### Phase 2: Type Check
 ```bash
@@ -100,13 +102,43 @@ Tests:     [PASS/FAIL] (X/Y passed, Z% coverage)
 Security:  [PASS/FAIL] (X issues)
 Diff:      [X files changed]
 
-Overall:   [READY/NOT READY] for PR
+Verdict:   PASS | PASS_WITH_NITS | FAIL
 
-Issues to Fix:
-1. ...
-2. ...
+Findings:
+| ID | Severity | Finding | Evidence (file:line / command output) |
+|----|----------|---------|---------------------------------------|
+| F1 | HIGH     | ...     | ...                                   |
 ```
 
+## Verdict Contract
+
+The report ends with exactly one verdict. Free-prose closings ("looks ready", "should be
+fine") are banned — they leave room to bury defects. A fixed vocabulary makes the report
+honest and machine-checkable.
+
+| Verdict | Meaning | Action |
+|---------|---------|--------|
+| **PASS** | All gates green, zero findings at any severity | Ship |
+| **PASS_WITH_NITS** | Ship-safe: only LOW/MEDIUM findings, each recorded with a follow-up | Ship + log follow-ups |
+| **FAIL** | Any gate red, or one or more CRITICAL/HIGH findings | Block → fix → **re-verify** |
+
+Every finding gets exactly one severity:
+
+- **CRITICAL** — data loss, security hole, or the change misbehaves in real use if shipped
+- **HIGH** — main-path defect or regression; users will hit it
+- **MEDIUM** — edge-case or quality defect; unlikely to block real use
+- **LOW** — nit: style, naming, doc wording
+
+Rules:
+- Severity is judged by impact evidence, not by how easy the fix is.
+- FAIL → fix → re-verify is one cycle. A fix alone never upgrades the verdict — the
+  re-verification must reproduce green.
+- A run that aborts early (Phase 1 build failure) still emits a report: verdict **FAIL**
+  with the failing gate as a CRITICAL finding. Stopping to fix is how you *reach* the next
+  verdict, not a reason to skip issuing this one — an unreported run reads as "not run".
+- The instance that wrote the change never issues its own verdict: verification runs in a
+  fresh instance (see the model-orchestration skill's V&V separation).
+
 ## Continuous Mode
 
 For long sessions, run verification every 15 minutes or after major changes: