@walwal-harness/cli 6.1.5 → 7.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/HR-Resource/academic-academic-anthropologist/SKILL.md +142 -0
- package/HR-Resource/academic-academic-geographer/SKILL.md +144 -0
- package/HR-Resource/academic-academic-historian/SKILL.md +140 -0
- package/HR-Resource/academic-academic-narratologist/SKILL.md +135 -0
- package/HR-Resource/academic-academic-psychologist/SKILL.md +135 -0
- package/HR-Resource/brick-office/SKILL.md +20 -0
- package/HR-Resource/cdo/SKILL.md +33 -0
- package/HR-Resource/ceo/SKILL.md +36 -0
- package/HR-Resource/coo/SKILL.md +36 -0
- package/HR-Resource/cqo/SKILL.md +33 -0
- package/HR-Resource/cto/SKILL.md +36 -0
- package/HR-Resource/design-design-brand-guardian/SKILL.md +339 -0
- package/HR-Resource/design-design-image-prompt-engineer/SKILL.md +253 -0
- package/HR-Resource/design-design-inclusive-visuals-specialist/SKILL.md +88 -0
- package/HR-Resource/design-design-ui-designer/SKILL.md +400 -0
- package/HR-Resource/design-design-ux-architect/SKILL.md +486 -0
- package/HR-Resource/design-design-ux-researcher/SKILL.md +346 -0
- package/HR-Resource/design-design-visual-storyteller/SKILL.md +166 -0
- package/HR-Resource/design-design-whimsy-injector/SKILL.md +455 -0
- package/HR-Resource/engineering-engineering-ai-data-remediation-engineer/SKILL.md +227 -0
- package/HR-Resource/engineering-engineering-ai-engineer/SKILL.md +163 -0
- package/HR-Resource/engineering-engineering-autonomous-optimization-architect/SKILL.md +124 -0
- package/HR-Resource/engineering-engineering-backend-architect/SKILL.md +252 -0
- package/HR-Resource/engineering-engineering-cms-developer/SKILL.md +553 -0
- package/HR-Resource/engineering-engineering-code-reviewer/SKILL.md +93 -0
- package/HR-Resource/engineering-engineering-codebase-onboarding-engineer/SKILL.md +190 -0
- package/HR-Resource/engineering-engineering-data-engineer/SKILL.md +323 -0
- package/HR-Resource/engineering-engineering-database-optimizer/SKILL.md +193 -0
- package/HR-Resource/engineering-engineering-devops-automator/SKILL.md +393 -0
- package/HR-Resource/engineering-engineering-email-intelligence-engineer/SKILL.md +370 -0
- package/HR-Resource/engineering-engineering-embedded-firmware-engineer/SKILL.md +190 -0
- package/HR-Resource/engineering-engineering-feishu-integration-developer/SKILL.md +615 -0
- package/HR-Resource/engineering-engineering-filament-optimization-specialist/SKILL.md +300 -0
- package/HR-Resource/engineering-engineering-frontend-developer/SKILL.md +242 -0
- package/HR-Resource/engineering-engineering-git-workflow-master/SKILL.md +101 -0
- package/HR-Resource/engineering-engineering-incident-response-commander/SKILL.md +461 -0
- package/HR-Resource/engineering-engineering-minimal-change-engineer/SKILL.md +224 -0
- package/HR-Resource/engineering-engineering-mobile-app-builder/SKILL.md +510 -0
- package/HR-Resource/engineering-engineering-rapid-prototyper/SKILL.md +479 -0
- package/HR-Resource/engineering-engineering-security-engineer/SKILL.md +321 -0
- package/HR-Resource/engineering-engineering-senior-developer/SKILL.md +193 -0
- package/HR-Resource/engineering-engineering-software-architect/SKILL.md +98 -0
- package/HR-Resource/engineering-engineering-solidity-smart-contract-engineer/SKILL.md +539 -0
- package/HR-Resource/engineering-engineering-sre/SKILL.md +107 -0
- package/HR-Resource/engineering-engineering-technical-writer/SKILL.md +410 -0
- package/HR-Resource/engineering-engineering-threat-detection-engineer/SKILL.md +551 -0
- package/HR-Resource/engineering-engineering-voice-ai-integration-engineer/SKILL.md +578 -0
- package/HR-Resource/engineering-engineering-wechat-mini-program-developer/SKILL.md +367 -0
- package/HR-Resource/finance-finance-bookkeeper-controller/SKILL.md +277 -0
- package/HR-Resource/finance-finance-financial-analyst/SKILL.md +251 -0
- package/HR-Resource/finance-finance-fpa-analyst/SKILL.md +280 -0
- package/HR-Resource/finance-finance-investment-researcher/SKILL.md +289 -0
- package/HR-Resource/finance-finance-tax-strategist/SKILL.md +256 -0
- package/HR-Resource/game-development-blender-blender-addon-engineer/SKILL.md +251 -0
- package/HR-Resource/game-development-game-audio-engineer/SKILL.md +281 -0
- package/HR-Resource/game-development-game-designer/SKILL.md +184 -0
- package/HR-Resource/game-development-godot-godot-gameplay-scripter/SKILL.md +351 -0
- package/HR-Resource/game-development-godot-godot-multiplayer-engineer/SKILL.md +314 -0
- package/HR-Resource/game-development-godot-godot-shader-developer/SKILL.md +283 -0
- package/HR-Resource/game-development-level-designer/SKILL.md +225 -0
- package/HR-Resource/game-development-narrative-designer/SKILL.md +260 -0
- package/HR-Resource/game-development-roblox-studio-roblox-avatar-creator/SKILL.md +314 -0
- package/HR-Resource/game-development-roblox-studio-roblox-experience-designer/SKILL.md +322 -0
- package/HR-Resource/game-development-roblox-studio-roblox-systems-scripter/SKILL.md +342 -0
- package/HR-Resource/game-development-technical-artist/SKILL.md +246 -0
- package/HR-Resource/game-development-unity-unity-architect/SKILL.md +288 -0
- package/HR-Resource/game-development-unity-unity-editor-tool-developer/SKILL.md +327 -0
- package/HR-Resource/game-development-unity-unity-multiplayer-engineer/SKILL.md +338 -0
- package/HR-Resource/game-development-unity-unity-shader-graph-artist/SKILL.md +286 -0
- package/HR-Resource/game-development-unreal-engine-unreal-multiplayer-architect/SKILL.md +330 -0
- package/HR-Resource/game-development-unreal-engine-unreal-systems-engineer/SKILL.md +327 -0
- package/HR-Resource/game-development-unreal-engine-unreal-technical-artist/SKILL.md +273 -0
- package/HR-Resource/game-development-unreal-engine-unreal-world-builder/SKILL.md +290 -0
- package/HR-Resource/hiring/SKILL.md +28 -0
- package/HR-Resource/index.json +747 -0
- package/HR-Resource/integrations-mcp-memory-backend-architect-with-memory/SKILL.md +264 -0
- package/HR-Resource/marketing-marketing-agentic-search-optimizer/SKILL.md +328 -0
- package/HR-Resource/marketing-marketing-ai-citation-strategist/SKILL.md +187 -0
- package/HR-Resource/marketing-marketing-app-store-optimizer/SKILL.md +338 -0
- package/HR-Resource/marketing-marketing-baidu-seo-specialist/SKILL.md +243 -0
- package/HR-Resource/marketing-marketing-bilibili-content-strategist/SKILL.md +216 -0
- package/HR-Resource/marketing-marketing-book-co-author/SKILL.md +127 -0
- package/HR-Resource/marketing-marketing-carousel-growth-engine/SKILL.md +216 -0
- package/HR-Resource/marketing-marketing-china-ecommerce-operator/SKILL.md +300 -0
- package/HR-Resource/marketing-marketing-china-market-localization-strategist/SKILL.md +300 -0
- package/HR-Resource/marketing-marketing-content-creator/SKILL.md +71 -0
- package/HR-Resource/marketing-marketing-cross-border-ecommerce/SKILL.md +276 -0
- package/HR-Resource/marketing-marketing-douyin-strategist/SKILL.md +166 -0
- package/HR-Resource/marketing-marketing-growth-hacker/SKILL.md +71 -0
- package/HR-Resource/marketing-marketing-instagram-curator/SKILL.md +130 -0
- package/HR-Resource/marketing-marketing-kuaishou-strategist/SKILL.md +240 -0
- package/HR-Resource/marketing-marketing-linkedin-content-creator/SKILL.md +230 -0
- package/HR-Resource/marketing-marketing-livestream-commerce-coach/SKILL.md +322 -0
- package/HR-Resource/marketing-marketing-podcast-strategist/SKILL.md +294 -0
- package/HR-Resource/marketing-marketing-private-domain-operator/SKILL.md +325 -0
- package/HR-Resource/marketing-marketing-reddit-community-builder/SKILL.md +140 -0
- package/HR-Resource/marketing-marketing-seo-specialist/SKILL.md +338 -0
- package/HR-Resource/marketing-marketing-short-video-editing-coach/SKILL.md +429 -0
- package/HR-Resource/marketing-marketing-social-media-strategist/SKILL.md +142 -0
- package/HR-Resource/marketing-marketing-tiktok-strategist/SKILL.md +142 -0
- package/HR-Resource/marketing-marketing-twitter-engager/SKILL.md +143 -0
- package/HR-Resource/marketing-marketing-video-optimization-specialist/SKILL.md +136 -0
- package/HR-Resource/marketing-marketing-wechat-official-account/SKILL.md +162 -0
- package/HR-Resource/marketing-marketing-weibo-strategist/SKILL.md +257 -0
- package/HR-Resource/marketing-marketing-xiaohongshu-specialist/SKILL.md +155 -0
- package/HR-Resource/marketing-marketing-zhihu-strategist/SKILL.md +179 -0
- package/HR-Resource/ops/SKILL.md +46 -0
- package/HR-Resource/paid-media-paid-media-auditor/SKILL.md +88 -0
- package/HR-Resource/paid-media-paid-media-creative-strategist/SKILL.md +88 -0
- package/HR-Resource/paid-media-paid-media-paid-social-strategist/SKILL.md +88 -0
- package/HR-Resource/paid-media-paid-media-ppc-strategist/SKILL.md +88 -0
- package/HR-Resource/paid-media-paid-media-programmatic-buyer/SKILL.md +88 -0
- package/HR-Resource/paid-media-paid-media-search-query-analyst/SKILL.md +88 -0
- package/HR-Resource/paid-media-paid-media-tracking-specialist/SKILL.md +88 -0
- package/HR-Resource/product-product-behavioral-nudge-engine/SKILL.md +97 -0
- package/HR-Resource/product-product-feedback-synthesizer/SKILL.md +136 -0
- package/HR-Resource/product-product-manager/SKILL.md +486 -0
- package/HR-Resource/product-product-sprint-prioritizer/SKILL.md +171 -0
- package/HR-Resource/product-product-trend-researcher/SKILL.md +176 -0
- package/HR-Resource/project-management-project-management-experiment-tracker/SKILL.md +215 -0
- package/HR-Resource/project-management-project-management-jira-workflow-steward/SKILL.md +247 -0
- package/HR-Resource/project-management-project-management-project-shepherd/SKILL.md +211 -0
- package/HR-Resource/project-management-project-management-studio-operations/SKILL.md +217 -0
- package/HR-Resource/project-management-project-management-studio-producer/SKILL.md +220 -0
- package/HR-Resource/project-management-project-manager-senior/SKILL.md +152 -0
- package/HR-Resource/resource-manager/SKILL.md +23 -0
- package/HR-Resource/sales-sales-account-strategist/SKILL.md +244 -0
- package/HR-Resource/sales-sales-coach/SKILL.md +288 -0
- package/HR-Resource/sales-sales-deal-strategist/SKILL.md +197 -0
- package/HR-Resource/sales-sales-discovery-coach/SKILL.md +242 -0
- package/HR-Resource/sales-sales-engineer/SKILL.md +199 -0
- package/HR-Resource/sales-sales-outbound-strategist/SKILL.md +218 -0
- package/HR-Resource/sales-sales-pipeline-analyst/SKILL.md +284 -0
- package/HR-Resource/sales-sales-proposal-strategist/SKILL.md +234 -0
- package/HR-Resource/spatial-computing-macos-spatial-metal-engineer/SKILL.md +354 -0
- package/HR-Resource/spatial-computing-terminal-integration-specialist/SKILL.md +87 -0
- package/HR-Resource/spatial-computing-visionos-spatial-engineer/SKILL.md +71 -0
- package/HR-Resource/spatial-computing-xr-cockpit-interaction-specialist/SKILL.md +49 -0
- package/HR-Resource/spatial-computing-xr-immersive-developer/SKILL.md +49 -0
- package/HR-Resource/spatial-computing-xr-interface-architect/SKILL.md +49 -0
- package/HR-Resource/specialized-accounts-payable-agent/SKILL.md +202 -0
- package/HR-Resource/specialized-agentic-identity-trust/SKILL.md +404 -0
- package/HR-Resource/specialized-agents-orchestrator/SKILL.md +384 -0
- package/HR-Resource/specialized-automation-governance-architect/SKILL.md +233 -0
- package/HR-Resource/specialized-blockchain-security-auditor/SKILL.md +480 -0
- package/HR-Resource/specialized-compliance-auditor/SKILL.md +175 -0
- package/HR-Resource/specialized-corporate-training-designer/SKILL.md +209 -0
- package/HR-Resource/specialized-customer-service/SKILL.md +415 -0
- package/HR-Resource/specialized-data-consolidation-agent/SKILL.md +77 -0
- package/HR-Resource/specialized-government-digital-presales-consultant/SKILL.md +380 -0
- package/HR-Resource/specialized-healthcare-customer-service/SKILL.md +406 -0
- package/HR-Resource/specialized-healthcare-marketing-compliance/SKILL.md +412 -0
- package/HR-Resource/specialized-hospitality-guest-services/SKILL.md +620 -0
- package/HR-Resource/specialized-hr-onboarding/SKILL.md +468 -0
- package/HR-Resource/specialized-identity-graph-operator/SKILL.md +277 -0
- package/HR-Resource/specialized-language-translator/SKILL.md +281 -0
- package/HR-Resource/specialized-legal-billing-time-tracking/SKILL.md +586 -0
- package/HR-Resource/specialized-legal-client-intake/SKILL.md +509 -0
- package/HR-Resource/specialized-legal-document-review/SKILL.md +471 -0
- package/HR-Resource/specialized-loan-officer-assistant/SKILL.md +572 -0
- package/HR-Resource/specialized-lsp-index-engineer/SKILL.md +331 -0
- package/HR-Resource/specialized-real-estate-buyer-seller/SKILL.md +613 -0
- package/HR-Resource/specialized-recruitment-specialist/SKILL.md +526 -0
- package/HR-Resource/specialized-report-distribution-agent/SKILL.md +82 -0
- package/HR-Resource/specialized-retail-customer-returns/SKILL.md +583 -0
- package/HR-Resource/specialized-sales-data-extraction-agent/SKILL.md +84 -0
- package/HR-Resource/specialized-sales-outreach/SKILL.md +442 -0
- package/HR-Resource/specialized-specialized-chief-of-staff/SKILL.md +296 -0
- package/HR-Resource/specialized-specialized-civil-engineer/SKILL.md +373 -0
- package/HR-Resource/specialized-specialized-cultural-intelligence-strategist/SKILL.md +105 -0
- package/HR-Resource/specialized-specialized-developer-advocate/SKILL.md +334 -0
- package/HR-Resource/specialized-specialized-document-generator/SKILL.md +72 -0
- package/HR-Resource/specialized-specialized-french-consulting-market/SKILL.md +209 -0
- package/HR-Resource/specialized-specialized-korean-business-navigator/SKILL.md +233 -0
- package/HR-Resource/specialized-specialized-mcp-builder/SKILL.md +265 -0
- package/HR-Resource/specialized-specialized-model-qa/SKILL.md +505 -0
- package/HR-Resource/specialized-specialized-salesforce-architect/SKILL.md +197 -0
- package/HR-Resource/specialized-specialized-workflow-architect/SKILL.md +614 -0
- package/HR-Resource/specialized-study-abroad-advisor/SKILL.md +299 -0
- package/HR-Resource/specialized-supply-chain-strategist/SKILL.md +599 -0
- package/HR-Resource/specialized-zk-steward/SKILL.md +228 -0
- package/HR-Resource/support-support-analytics-reporter/SKILL.md +382 -0
- package/HR-Resource/support-support-executive-summary-generator/SKILL.md +229 -0
- package/HR-Resource/support-support-finance-tracker/SKILL.md +459 -0
- package/HR-Resource/support-support-infrastructure-maintainer/SKILL.md +635 -0
- package/HR-Resource/support-support-legal-compliance-checker/SKILL.md +605 -0
- package/HR-Resource/support-support-support-responder/SKILL.md +602 -0
- package/HR-Resource/testing-testing-accessibility-auditor/SKILL.md +333 -0
- package/HR-Resource/testing-testing-api-tester/SKILL.md +323 -0
- package/HR-Resource/testing-testing-evidence-collector/SKILL.md +227 -0
- package/HR-Resource/testing-testing-performance-benchmarker/SKILL.md +285 -0
- package/HR-Resource/testing-testing-reality-checker/SKILL.md +253 -0
- package/HR-Resource/testing-testing-test-results-analyzer/SKILL.md +322 -0
- package/HR-Resource/testing-testing-tool-evaluator/SKILL.md +411 -0
- package/HR-Resource/testing-testing-workflow-optimizer/SKILL.md +467 -0
- package/assets/launchd/com.walwal.harness-wake.plist.template +1 -1
- package/assets/templates/company-pipeline-manifest.json +71 -0
- package/assets/templates/config.json +40 -2
- package/bin/init.js +208 -109
- package/commands/goal.md +25 -0
- package/commands/hot-fix.md +25 -0
- package/commands/play-harness.md +22 -0
- package/commands/release-harness.md +22 -0
- package/commands/stop-harness.md +21 -0
- package/conventions/README.md +14 -88
- package/conventions/brick-office.md +5 -0
- package/conventions/cdo.md +6 -0
- package/conventions/ceo.md +9 -0
- package/conventions/coo.md +6 -0
- package/conventions/cqo.md +6 -23
- package/conventions/cto.md +5 -23
- package/conventions/hiring.md +7 -0
- package/conventions/ops.md +9 -0
- package/conventions/resource-manager.md +6 -0
- package/conventions/shared.md +29 -40
- package/gotchas/README.md +14 -82
- package/gotchas/brick-office.md +9 -0
- package/gotchas/cdo.md +9 -0
- package/gotchas/ceo.md +9 -0
- package/gotchas/coo.md +9 -0
- package/gotchas/cqo.md +6 -19
- package/gotchas/cto.md +6 -19
- package/gotchas/hiring.md +9 -0
- package/gotchas/ops.md +17 -0
- package/gotchas/resource-manager.md +9 -0
- package/gotchas/shared.md +13 -0
- package/package.json +11 -17
- package/scripts/conductor-tick.sh +97 -4
- package/scripts/harness-hourly-review.sh +284 -5
- package/scripts/harness-parity-audit.sh +101 -0
- package/scripts/harness-queue-manager.sh +1 -1
- package/scripts/harness-runtime-stop.sh +58 -0
- package/scripts/harness-service-ops-monitor.sh +207 -18
- package/scripts/harness-stop.sh +18 -0
- package/scripts/harness-wake-install.sh +1 -0
- package/scripts/harness-wake.sh +100 -8
- package/scripts/harness-worker-dispatch.sh +52 -21
- package/scripts/harness-worker-evidence-validate.sh +59 -0
- package/scripts/import-agency-agents.js +102 -0
- package/scripts/lib/harness-agent-resolver.sh +127 -0
- package/scripts/play.sh +104 -0
- package/scripts/release.sh +89 -0
- package/conventions/conductor.md +0 -24
- package/conventions/coo-developer.md +0 -24
- package/conventions/dispatcher.md +0 -24
- package/conventions/documentationer.md +0 -24
- package/conventions/evaluator-architecture.md +0 -24
- package/conventions/evaluator-code-quality.md +0 -24
- package/conventions/evaluator-functional.md +0 -24
- package/conventions/evaluator-security.md +0 -24
- package/conventions/evaluator-visual.md +0 -24
- package/conventions/generator-backend.md +0 -24
- package/conventions/generator-designer.md +0 -24
- package/conventions/generator-devops.md +0 -24
- package/conventions/generator-frontend.md +0 -24
- package/conventions/meeting-manager.md +0 -24
- package/conventions/planner.md +0 -24
- package/conventions/service-ops.md +0 -24
- package/gotchas/conductor.md +0 -38
- package/gotchas/coo-developer.md +0 -22
- package/gotchas/dispatcher.md +0 -38
- package/gotchas/documentationer.md +0 -22
- package/gotchas/evaluator-architecture.md +0 -22
- package/gotchas/evaluator-code-quality.md +0 -22
- package/gotchas/evaluator-functional.md +0 -22
- package/gotchas/evaluator-security.md +0 -22
- package/gotchas/evaluator-visual.md +0 -22
- package/gotchas/generator-backend-laravel.md +0 -19
- package/gotchas/generator-backend.md +0 -22
- package/gotchas/generator-designer.md +0 -22
- package/gotchas/generator-devops.md +0 -22
- package/gotchas/generator-frontend.md +0 -22
- package/gotchas/meeting-manager.md +0 -30
- package/gotchas/planner.md +0 -22
- package/gotchas/service-ops.md +0 -30
- package/skills/_shared/dynamic-registration.md +0 -120
- package/skills/brainstorming/SKILL.md +0 -220
- package/skills/brainstorming/references/attribution.md +0 -109
- package/skills/brainstorming/references/spec-document-reviewer-prompt.md +0 -49
- package/skills/brainstorming/references/visual-companion.md +0 -287
- package/skills/brainstorming/scripts/frame-template.html +0 -214
- package/skills/brainstorming/scripts/helper.js +0 -88
- package/skills/brainstorming/scripts/server.cjs +0 -354
- package/skills/brainstorming/scripts/start-server.sh +0 -148
- package/skills/brainstorming/scripts/stop-server.sh +0 -56
- package/skills/conductor/SKILL.md +0 -336
- package/skills/coo-developer/SKILL.md +0 -72
- package/skills/cqo/SKILL.md +0 -146
- package/skills/cto/SKILL.md +0 -141
- package/skills/dispatcher/SKILL.md +0 -298
- package/skills/dispatcher/persona-ceo.md +0 -169
- package/skills/dispatcher/references/convention-flow.md +0 -111
- package/skills/dispatcher/references/gotcha-flow.md +0 -79
- package/skills/dispatcher/references/initialization.md +0 -53
- package/skills/dispatcher/references/pipeline-definitions.md +0 -151
- package/skills/documentationer/SKILL.md +0 -92
- package/skills/evaluator-architecture/SKILL.md +0 -173
- package/skills/evaluator-code-quality/SKILL.md +0 -218
- package/skills/evaluator-code-quality/references/scoring-rubric.md +0 -104
- package/skills/evaluator-functional/SKILL.md +0 -271
- package/skills/evaluator-functional/references/ia-compliance.md +0 -37
- package/skills/evaluator-functional/references/playwright-tools.md +0 -43
- package/skills/evaluator-functional/references/scoring-rubric.md +0 -52
- package/skills/evaluator-security/SKILL.md +0 -172
- package/skills/evaluator-visual/SKILL.md +0 -165
- package/skills/evaluator-visual/references/responsive-checklist.md +0 -44
- package/skills/evaluator-visual/references/scoring-rubric.md +0 -59
- package/skills/generator-backend/SKILL.md +0 -153
- package/skills/generator-backend/references/nestjs-msa-patterns.md +0 -69
- package/skills/generator-backend/references/sprint-contract-be.md +0 -32
- package/skills/generator-designer/SKILL.md +0 -219
- package/skills/generator-devops/SKILL.md +0 -201
- package/skills/generator-frontend/SKILL.md +0 -155
- package/skills/generator-frontend/references/_web-react-legacy/ai-forbidden-patterns.md +0 -89
- package/skills/generator-frontend/references/_web-react-legacy/component-patterns.md +0 -52
- package/skills/generator-frontend/references/_web-react-legacy/design-system-rules.md +0 -139
- package/skills/generator-frontend/references/_web-react-legacy/vercel-best-practices.md +0 -54
- package/skills/meeting-manager/SKILL.md +0 -419
- package/skills/planner/SKILL.md +0 -243
- package/skills/planner/hr-onboard.md +0 -134
- package/skills/planner/hr-recruit.md +0 -99
- package/skills/planner/persona-coo-hr.md +0 -182
- package/skills/planner/references/api-contract-schema.md +0 -34
- package/skills/planner/references/fe-stack-detection.md +0 -164
- package/skills/planner/references/ia-map-guide.md +0 -31
- package/skills/planner/references/plan-template.md +0 -51
- package/skills/service-ops/SKILL.md +0 -295
|
@@ -1,271 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: harness-evaluator-functional
|
|
3
|
-
description: "하네스 Functional Evaluator. Playwright MCP(browser_*)로 실행 중인 앱을 실제 사용자처럼 조작하며 E2E 기능을 검증한다. Step 0 IA 구조 검증(Gate) → Step 1-7 기능 테스트. 기준 미달 = FAIL."
|
|
4
|
-
disable-model-invocation: false
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
# Evaluator-Functional — Playwright MCP
|
|
8
|
-
|
|
9
|
-
## progress.json 업데이트 규칙 (v5.6.3+)
|
|
10
|
-
|
|
11
|
-
⚠️ **절대로 progress.json 을 통째로 재작성하지 마라**. `Write` 도구로 전체 파일을
|
|
12
|
-
덮어쓰면 `mode` / `company_state` / 기타 top-level 필드가 누락되어 회사모드 병렬 루프가
|
|
13
|
-
끊기는 등 런타임 오류가 발생한다.
|
|
14
|
-
|
|
15
|
-
**올바른 방법** — 반드시 partial update 로 갱신:
|
|
16
|
-
|
|
17
|
-
```bash
|
|
18
|
-
# 헬퍼 스크립트 (권장)
|
|
19
|
-
bash scripts/harness-progress-set.sh . '.current_agent = "planner" | .agent_status = "running"'
|
|
20
|
-
|
|
21
|
-
# 또는 직접 jq 로 partial update
|
|
22
|
-
jq '.agent_status = "completed" | .completed_agents += ["planner"]' .harness/progress.json > .harness/progress.json.tmp && mv .harness/progress.json.tmp .harness/progress.json
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
위 두 방식은 파일의 나머지 필드를 보존한다. Read → 수정 → Write 패턴은 사용 금지.
|
|
26
|
-
|
|
27
|
-
## Session Boundary Protocol
|
|
28
|
-
|
|
29
|
-
### On Start
|
|
30
|
-
1. `.harness/progress.json` 읽기 — `next_agent`가 `"evaluator-functional"`인지 확인
|
|
31
|
-
2. progress.json 업데이트: `current_agent` → `"evaluator-functional"`, `agent_status` → `"running"`, `updated_at` 갱신
|
|
32
|
-
|
|
33
|
-
### On Complete (PASS)
|
|
34
|
-
1. **Screenshot Cleanup** — 이번 평가에서 `browser_take_screenshot` 으로 생성한 모든 PNG/JPEG 파일 삭제:
|
|
35
|
-
```bash
|
|
36
|
-
find . -maxdepth 3 \( -name "screenshot*.png" -o -name "screenshot*.jpg" -o -name "playwright-*.png" \) -newer .harness/progress.json -delete 2>/dev/null
|
|
37
|
-
```
|
|
38
|
-
증거는 `evaluation-functional.md` 에 텍스트로 기술 — 파일은 남기지 않는다.
|
|
39
|
-
2. progress.json 업데이트:
|
|
40
|
-
- `agent_status` → `"completed"`
|
|
41
|
-
- `completed_agents`에 `"evaluator-functional"` 추가
|
|
42
|
-
- `next_agent` → 파이프라인에 따라 결정 (FULLSTACK/FE-ONLY: `"evaluator-visual"`, BE-ONLY: `"archive"`)
|
|
43
|
-
- `failure` 필드 초기화
|
|
44
|
-
3. `feature-list.json`의 통과 feature `passes`에 `"evaluator-functional"` 추가
|
|
45
|
-
4. `.harness/progress.log`에 PASS 요약 추가
|
|
46
|
-
5. 출력: `"✓ Evaluator-Functional PASS. 자동 핸드오프 (Conductor 자율 시동)."`
|
|
47
|
-
6. **즉시 내부 핸드오프 (scripts/harness-next.sh) 를 호출하여 다음 에이전트로 자동 핸드오프** (회사모드 내부 핸드오프).
|
|
48
|
-
|
|
49
|
-
### On Fail
|
|
50
|
-
1. **Screenshot Cleanup** — PASS 와 동일하게 스크린샷 파일 삭제 (FAIL 시에도 정리 필수).
|
|
51
|
-
2. progress.json 업데이트:
|
|
52
|
-
- `agent_status` → `"failed"`
|
|
53
|
-
- `failure.agent` → `"evaluator-functional"`
|
|
54
|
-
- `failure.location` → `"backend"` 또는 `"frontend"` (결함 위치)
|
|
55
|
-
- `failure.message` → 실패 요약 (1줄)
|
|
56
|
-
- `failure.retry_target` → `"generator-backend"` 또는 `"generator-frontend"`
|
|
57
|
-
- `next_agent` → `failure.retry_target`과 동일
|
|
58
|
-
- `sprint.retry_count` 증가
|
|
59
|
-
3. `sprint.retry_count >= 10`이면 `agent_status` → `"blocked"`, 사용자 개입 요청
|
|
60
|
-
4. `.harness/progress.log`에 FAIL 요약 추가
|
|
61
|
-
5. 출력: `"✖ Evaluator-Functional FAIL. 자동 핸드오프 (Conductor 자율 시동) (재작업 대상으로 라우팅)."`
|
|
62
|
-
6. **즉시 내부 핸드오프 (scripts/harness-next.sh) 를 호출하여 `failure.retry_target` 으로 자동 핸드오프** (회사모드 내부 핸드오프).
|
|
63
|
-
|
|
64
|
-
## Critical Mindset
|
|
65
|
-
|
|
66
|
-
- **회의적 평가자**. Generator의 자체 평가를 신뢰하지 마세요.
|
|
67
|
-
- 문제 발견 후 "사소하다"고 자기 설득 금지.
|
|
68
|
-
- 코드 읽기는 평가가 아님 — **반드시 앱을 조작**.
|
|
69
|
-
- 기준 미달 = FAIL. 예외 없음.
|
|
70
|
-
|
|
71
|
-
## FE Playwright Mandatory Rule (v5.4)
|
|
72
|
-
|
|
73
|
-
**프론트엔드 Feature(FE-ONLY 또는 FULLSTACK의 FE 부분)는 반드시 Playwright MCP 도구 호출로 검증**한다. 예외 없음.
|
|
74
|
-
|
|
75
|
-
- 필수 호출 도구 (최소 1회 이상): `mcp__playwright__browser_navigate`, `mcp__playwright__browser_snapshot` 또는 `mcp__playwright__browser_take_screenshot`, 그리고 AC 검증을 위한 interaction (`browser_click`, `browser_type`, `browser_fill_form`, `browser_evaluate` 등).
|
|
76
|
-
- **금지**: 소스 코드 열람, grep, 정적 분석만으로 FE Feature를 PASS 처리하는 것.
|
|
77
|
-
- **금지**: "dev 서버 기동 실패"로 Playwright 단계를 스킵하는 것. 서버 기동까지 Evaluator의 책임.
|
|
78
|
-
- `evaluation-functional.md`에 호출한 **playwright 도구 이름 + 결과 요약**을 AC별로 기술. 도구 호출 증거 없으면 해당 AC는 자동 0점(Evidence 없는 Score = 0점 강제 규칙).
|
|
79
|
-
- BE-ONLY Feature는 이 규칙 대상 아님 (CLI 기반 API 테스트 유지).
|
|
80
|
-
|
|
81
|
-
## Startup
|
|
82
|
-
|
|
83
|
-
1. `AGENTS.md` 읽기 — IA-MAP
|
|
84
|
-
2. `CONVENTIONS.md` (루트) 읽기 — 프로젝트 최상위 원칙 (있을 때만)
|
|
85
|
-
3. `.harness/conventions/shared.md` + `.harness/conventions/evaluator-functional.md` — **긍정 하우스 스타일 적용**
|
|
86
|
-
4. `.harness/gotchas/evaluator-functional.md` 읽기 — **과거 실수 반복 금지**
|
|
87
|
-
5. `.harness/memory.md` 읽기 — **프로젝트 공유 학습 규칙 적용**
|
|
88
|
-
6. `actions/sprint-contract.md` — BE + FE 성공 기준 전체
|
|
89
|
-
7. `actions/feature-list.json` — 이번 스프린트 범위
|
|
90
|
-
8. `actions/api-contract.json` — 기대 API 동작
|
|
91
|
-
9. `.harness/progress.json`
|
|
92
|
-
|
|
93
|
-
## Feature-Level Company Worker
|
|
94
|
-
|
|
95
|
-
Company worker가 호출할 때, 프롬프트에 `FEATURE_ID`가 지정된다.
|
|
96
|
-
|
|
97
|
-
### Feature-Level Rules
|
|
98
|
-
- `feature-list.json`에서 **지정된 FEATURE_ID의 AC만** 검증
|
|
99
|
-
- Regression: `feature-queue.json`의 `passed` 목록에 있는 Feature들의 AC 재검증
|
|
100
|
-
- Cross-Validation: Feature 단위에서는 skip (Sprint-End에서 수행)
|
|
101
|
-
- Visual Evaluation: Feature 단위에서는 skip (Sprint-End에서 수행)
|
|
102
|
-
- 출력 형식: `---EVAL-RESULT---` 블록 (Worker가 파싱 가능)
|
|
103
|
-
|
|
104
|
-
### Feature-Level Scoring
|
|
105
|
-
- 동일한 R1-R5 루브릭 적용
|
|
106
|
-
- PASS 기준: 2.80/3.00 (변경 없음)
|
|
107
|
-
- 1건이라도 Regression 실패 시 FAIL (변경 없음)
|
|
108
|
-
|
|
109
|
-
### Output Format (Machine-Parseable)
|
|
110
|
-
```
|
|
111
|
-
---EVAL-RESULT---
|
|
112
|
-
FEATURE: F-XXX
|
|
113
|
-
VERDICT: PASS or FAIL
|
|
114
|
-
SCORE: X.XX
|
|
115
|
-
FEEDBACK: one paragraph summary
|
|
116
|
-
---END-EVAL-RESULT---
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
## Stack-Adaptive Validation (v5.2)
|
|
120
|
-
|
|
121
|
-
Evaluator 는 스택마다 다른 검증 도구를 가진다. `scan-result.json.tech_stack` 에서 현재 스택을 확인한 뒤 `.harness/ref/<role>-<stack>.md` 의 `validation` 블록을 로드해 순차 실행한다.
|
|
122
|
-
|
|
123
|
-
### Validation 블록 파싱
|
|
124
|
-
|
|
125
|
-
```
|
|
126
|
-
1. ref-docs YAML frontmatter 파싱 → validation 객체 추출
|
|
127
|
-
2. validation.pre_eval_gate 의 모든 명령을 순차 실행
|
|
128
|
-
- 실패 시 → FAIL + generator 로 retry (Pre-Eval Gate)
|
|
129
|
-
3. validation.functional_tests 의 모든 명령을 순차 실행
|
|
130
|
-
- 실패 시 → FAIL 항목 기록
|
|
131
|
-
4. validation.anti_pattern_rules 순회:
|
|
132
|
-
- pattern_type == "grep": `grep -rE "<pattern>" <paths>` 로 스캔
|
|
133
|
-
- pattern_type == "lint_tool": `<tool> <args>` 로 호출 + JSON 출력 파싱
|
|
134
|
-
- 위반 발견 시 → Auto Gotcha Registration (아래)
|
|
135
|
-
5. validation.visual.enabled:
|
|
136
|
-
- true → evaluator-visual 에 Playwright 검증 위임
|
|
137
|
-
- false → evaluation-functional.md 에 "MANUAL_REQUIRED: {manual_check}" 기록, Visual 은 __skip__
|
|
138
|
-
```
|
|
139
|
-
|
|
140
|
-
### Auto Gotcha Registration (안티패턴 + 평가 실패 자동 등록) — v5.7.1+
|
|
141
|
-
|
|
142
|
-
**필수 emission**: `actions/evaluation-functional.md` 끝부분에 아래 fenced JSON 블록을 반드시 포함한다 (후보 없으면 빈 배열 `[]`). `harness-next.sh` 가 Evaluator 완료 직후 이 블록을 스캔해 `scripts/harness-gotcha-register.sh` 로 자동 등록한다.
|
|
143
|
-
|
|
144
|
-
````
|
|
145
|
-
```gotcha_candidates
|
|
146
|
-
[
|
|
147
|
-
{
|
|
148
|
-
"target": "generator-frontend",
|
|
149
|
-
"rule_id": "fe-console-error-ignored",
|
|
150
|
-
"title": "콘솔 JS 에러 방치",
|
|
151
|
-
"wrong": "렌더 직후 발생하는 TypeError 를 수정하지 않고 PASS 주장",
|
|
152
|
-
"right": "콘솔 JS 에러 0 이 될 때까지 수정 후 재제출",
|
|
153
|
-
"why": "Evaluator-Functional 콘솔 청결 축은 15% 가중 하드 임계 (0개)",
|
|
154
|
-
"scope": "모든 FE Feature",
|
|
155
|
-
"source": "evaluator-functional:F-003"
|
|
156
|
-
}
|
|
157
|
-
]
|
|
158
|
-
```
|
|
159
|
-
````
|
|
160
|
-
|
|
161
|
-
등록 규칙:
|
|
162
|
-
- `target`: 실수를 반복할 **대상 에이전트** (예: `generator-frontend`, `generator-backend`, `planner`). 본인(`evaluator-*`) 대상도 가능.
|
|
163
|
-
- `rule_id`: 전역 유일 식별자. 동일 rule_id 는 Occurrences 증가 + Last-Seen 갱신 (본문 미변경).
|
|
164
|
-
- `source`: 출처 — `<agent>:<feature-id>` 형식 권장.
|
|
165
|
-
- 신규 항목은 `Status: unverified` 로 기록. Planner 리뷰 후 수동으로 `verified` 승격.
|
|
166
|
-
- 대상 파일: `.harness/gotchas/<target>.md` (스택별 파일 필요 시 `<target>-<stack>.md` 를 `target` 에 명시).
|
|
167
|
-
|
|
168
|
-
**FAIL 시**: 실패 근본 원인 1건 이상을 반드시 candidate 로 등록 (중복 실수 방지가 목적).
|
|
169
|
-
**PASS 시**: 발견된 안티패턴/경미한 위반이 있으면 등록 (스코어에는 반영 안 됐지만 반복 방지).
|
|
170
|
-
|
|
171
|
-
Dispatcher 경유 수동 등록은 여전히 가능하지만, **이 자동 파이프라인이 기본 경로**다.
|
|
172
|
-
|
|
173
|
-
## Evaluation Steps
|
|
174
|
-
|
|
175
|
-
### Step 0: IA Structure Compliance (GATE)
|
|
176
|
-
|
|
177
|
-
AGENTS.md IA-MAP vs 실제 구조 대조. **미통과 시 이하 전체 SKIP, 즉시 FAIL.**
|
|
178
|
-
|
|
179
|
-
상세 → [IA 검증 가이드](references/ia-compliance.md)
|
|
180
|
-
|
|
181
|
-
### Step 1-7: 기능 테스트
|
|
182
|
-
|
|
183
|
-
1. Environment Verification (브라우저 로드, 콘솔 에러)
|
|
184
|
-
2. API Health Check (Gateway 직접 검증)
|
|
185
|
-
3. Regression Test (이전 기능 재확인)
|
|
186
|
-
4. Contract Criteria Verification (각 기준 순서대로)
|
|
187
|
-
5. API Contract Compliance (api-contract.json 대조)
|
|
188
|
-
6. Error Scenario Testing
|
|
189
|
-
7. Console Error Audit
|
|
190
|
-
|
|
191
|
-
Playwright 도구 → [도구 레퍼런스](references/playwright-tools.md)
|
|
192
|
-
채점 기준 → [스코어링 루브릭](references/scoring-rubric.md)
|
|
193
|
-
|
|
194
|
-
## Scoring
|
|
195
|
-
|
|
196
|
-
| 차원 | 가중치 | 하드 임계값 |
|
|
197
|
-
|------|--------|------------|
|
|
198
|
-
| Contract 충족률 | 40% | 80% |
|
|
199
|
-
| API 계약 준수 | 25% | 100% |
|
|
200
|
-
| 에러 내성 | 20% | 6/10 |
|
|
201
|
-
| 콘솔 청결 | 15% | JS에러 0개 |
|
|
202
|
-
|
|
203
|
-
## After Evaluation
|
|
204
|
-
|
|
205
|
-
- **PASS** → Session Boundary Protocol On Complete (PASS) 실행
|
|
206
|
-
- **FAIL** → Session Boundary Protocol On Fail 실행
|
|
207
|
-
|
|
208
|
-
## ⚠ MANDATORY — 동적 Gotcha / Convention 등록 (모든 평가에서 필수)
|
|
209
|
-
|
|
210
|
-
evaluation-functional.md 의 끝에 **반드시 두 개의 fenced JSON 블록**을 작성한다. 비어 있어도 `[]` 로 명시 (블록 자체를 생략 금지). harness-next.sh 가 평가 직후 자동으로 이 블록들을 파싱하여 `.harness/gotchas/<target>.md` 와 `.harness/conventions/<scope>.md` 에 dedup append 한다.
|
|
211
|
-
|
|
212
|
-
**왜 mandatory 인가**: 사용자 명시 — "Gen/Eval 이 발견한 패턴은 메뉴얼로 시키기 전에 동적으로 등록되어야 한다. 자동으로 패턴화되는 이슈는 등록하라." 한 번 발견된 실수가 다음 sprint 에서 반복되지 않게 하려면 평가자가 그 자리에서 등록하는 것이 유일한 closure.
|
|
213
|
-
|
|
214
|
-
### 1) gotcha_candidates — 부정 가이드 (실수 패턴)
|
|
215
|
-
|
|
216
|
-
평가 중 발견한 **반복 가능한 실수** (한 번이라도 동일 패턴이 다시 나올 위험이 있는 결함). 한 평가에서 0~N 개.
|
|
217
|
-
|
|
218
|
-
검출 기준 (하나라도 해당하면 등록):
|
|
219
|
-
- 같은 sprint 내 다른 feature 에서도 재발할 가능성이 있는 결함
|
|
220
|
-
- generator 가 자주 빠뜨리는 케이스 (e.g. RSC↔CC 경계, schema 누락)
|
|
221
|
-
- spec/contract 위반 패턴
|
|
222
|
-
- AC 부분 통과 / Hard Gate 위반 사유
|
|
223
|
-
|
|
224
|
-
```gotcha_candidates
|
|
225
|
-
[
|
|
226
|
-
{
|
|
227
|
-
"target": "generator-frontend",
|
|
228
|
-
"rule_id": "rsc-cc-boundary-monitoring",
|
|
229
|
-
"title": "Server Component 에서 useState/useEffect 호출",
|
|
230
|
-
"wrong": "app/monitoring/page.tsx 에 'use client' 없이 hook 사용 → 빌드 통과해도 런타임에 깨짐",
|
|
231
|
-
"right": "client-side state 사용 시 파일 최상단에 'use client' 명시. 또는 server component 로 유지하면서 client 부분만 분리.",
|
|
232
|
-
"why": "Next.js App Router 의 RSC↔CC 경계는 정적 분석으로 100% 안 잡힘. F-209 에서 발견.",
|
|
233
|
-
"scope": "app/**/page.tsx, app/**/layout.tsx",
|
|
234
|
-
"source": "evaluator-functional:F-209"
|
|
235
|
-
}
|
|
236
|
-
]
|
|
237
|
-
```
|
|
238
|
-
|
|
239
|
-
비어 있으면:
|
|
240
|
-
```gotcha_candidates
|
|
241
|
-
[]
|
|
242
|
-
```
|
|
243
|
-
|
|
244
|
-
### 2) convention_candidates — 긍정 가이드 (반복 가능한 best practice)
|
|
245
|
-
|
|
246
|
-
평가 중 확립된 **반복 가능한 모범 사례** (다른 feature 에서도 같은 방식으로 적용해야 하는 룰). 한 평가에서 0~N 개.
|
|
247
|
-
|
|
248
|
-
검출 기준:
|
|
249
|
-
- 같은 sprint 의 다른 feature 가 모방해야 할 패턴
|
|
250
|
-
- API 계약 / 폴더 구조 / 명명 규칙 관련 결정
|
|
251
|
-
- 사용자가 "이렇게 해" 라고 한 한 번의 발언이 평가에서 일반화 가능한 경우
|
|
252
|
-
|
|
253
|
-
```convention_candidates
|
|
254
|
-
[
|
|
255
|
-
{
|
|
256
|
-
"scope": "generator-frontend",
|
|
257
|
-
"rule_id": "route-segment-files",
|
|
258
|
-
"title": "App Router 세그먼트 필수 파일 세트",
|
|
259
|
-
"rule": "모든 app/**/page.tsx 는 같은 폴더에 not-found.tsx, error.tsx, loading.tsx 를 함께 배치한다.",
|
|
260
|
-
"why": "Next.js 의 segment-level 에러/로딩 처리. 누락 시 default 흰 화면 노출. F-209 평가에서 표준화 결정.",
|
|
261
|
-
"source": "evaluator-functional:F-209"
|
|
262
|
-
}
|
|
263
|
-
]
|
|
264
|
-
```
|
|
265
|
-
|
|
266
|
-
비어 있으면:
|
|
267
|
-
```convention_candidates
|
|
268
|
-
[]
|
|
269
|
-
```
|
|
270
|
-
|
|
271
|
-
**금지**: 두 블록 중 하나라도 누락된 채로 평가 종료 → On Complete protocol 위반. harness-next.sh 의 audit gate 에서 누락 감지 시 경고.
|
|
@@ -1,37 +0,0 @@
|
|
|
1
|
-
# IA Structure Compliance — Step 0 (Gate)
|
|
2
|
-
|
|
3
|
-
## 검증 방법
|
|
4
|
-
|
|
5
|
-
```bash
|
|
6
|
-
# 1. 실제 폴더 구조 확인
|
|
7
|
-
ls -R apps/ libs/ 2>/dev/null
|
|
8
|
-
|
|
9
|
-
# 2. git diff로 소유권 위반 검출
|
|
10
|
-
git log --name-only --pretty=format: HEAD~[sprint_commits].. | sort -u
|
|
11
|
-
```
|
|
12
|
-
|
|
13
|
-
## 검증 항목
|
|
14
|
-
|
|
15
|
-
| 검증 | 판정 | 예시 |
|
|
16
|
-
|------|------|------|
|
|
17
|
-
| IA-MAP 경로가 실제 존재하는가 | 누락 → FAIL | apps/service-a/ 미생성 |
|
|
18
|
-
| IA-MAP에 없는 경로가 생겼는가 | 미등록 → DRIFT 기록 | apps/service-c/ 무단 생성 |
|
|
19
|
-
| [BE] 소유를 FE가 수정했는가 | 침범 → FAIL | apps/gateway/ FE 수정 |
|
|
20
|
-
| [FE] 소유를 BE가 수정했는가 | 침범 → FAIL | apps/web/ BE 수정 |
|
|
21
|
-
| [META]/[HARNESS]를 Generator가 수정했는가 | 침범 → FAIL | AGENTS.md 수정 |
|
|
22
|
-
|
|
23
|
-
## 판정 규칙
|
|
24
|
-
|
|
25
|
-
- **경로 누락 / 소유권 침범** → 즉시 FAIL, Step 1 이하 SKIP
|
|
26
|
-
- **미등록 경로 (Drift)** → FAIL 아님, evaluation에 `## AGENTS.md Drift` 기록
|
|
27
|
-
|
|
28
|
-
## Output (evaluation-functional.md에 포함)
|
|
29
|
-
|
|
30
|
-
```markdown
|
|
31
|
-
## Step 0: IA Structure Compliance
|
|
32
|
-
- Verdict: PASS / FAIL (GATE)
|
|
33
|
-
- IA-MAP paths checked: [N]개
|
|
34
|
-
- Missing paths: [목록 또는 "none"]
|
|
35
|
-
- Unregistered paths: [목록 또는 "none"]
|
|
36
|
-
- Ownership violations: [목록 또는 "none"]
|
|
37
|
-
```
|
|
@@ -1,43 +0,0 @@
|
|
|
1
|
-
# Playwright MCP Tools Reference
|
|
2
|
-
|
|
3
|
-
## 핵심 도구
|
|
4
|
-
|
|
5
|
-
| 도구 | 용도 | 주요 사용 Step |
|
|
6
|
-
|------|------|---------------|
|
|
7
|
-
| `browser_navigate` | URL 이동 | Step 1, 3, 4 |
|
|
8
|
-
| `browser_click` | 요소 클릭 | Step 3, 4 |
|
|
9
|
-
| `browser_fill` | 입력 필드 작성 | Step 4, 6 |
|
|
10
|
-
| `browser_select_option` | 드롭다운 선택 | Step 4 |
|
|
11
|
-
| `browser_press_key` | 키보드 (Enter, Escape, Tab) | Step 4, 6 |
|
|
12
|
-
| `browser_take_screenshot` | 스크린샷 (증거) | 모든 Step |
|
|
13
|
-
| `browser_snapshot` | 접근성 트리 (DOM 구조) | Step 1, 4 |
|
|
14
|
-
| `browser_console_messages` | 콘솔 에러 감지 | Step 1, 7 |
|
|
15
|
-
| `browser_network_requests` | API 호출 캡처 | Step 2, 4, 5 |
|
|
16
|
-
| `browser_wait` | 요소/상태 대기 | Step 4 |
|
|
17
|
-
| `browser_resize` | 뷰포트 크기 변경 | Step 4 |
|
|
18
|
-
| `browser_tabs` | 탭 목록 | Step 2 |
|
|
19
|
-
| `browser_handle_dialog` | alert/confirm 처리 | Step 6 |
|
|
20
|
-
| `browser_hover` | 호버 상태 | Step 4 |
|
|
21
|
-
| `browser_drag` | 드래그 앤 드롭 | Step 4 |
|
|
22
|
-
|
|
23
|
-
## 기준 검증 패턴
|
|
24
|
-
|
|
25
|
-
```
|
|
26
|
-
기준: "사용자가 아이템을 생성할 수 있다"
|
|
27
|
-
|
|
28
|
-
[Action]
|
|
29
|
-
1. browser_navigate → /items
|
|
30
|
-
2. browser_click → "새 아이템" 버튼
|
|
31
|
-
3. browser_fill → name 필드에 "Test"
|
|
32
|
-
4. browser_click → "저장"
|
|
33
|
-
5. browser_wait → 목록에 "Test" 표시
|
|
34
|
-
|
|
35
|
-
[Verify]
|
|
36
|
-
6. browser_take_screenshot → 결과 캡처
|
|
37
|
-
7. browser_network_requests → POST /api/v1/items 확인
|
|
38
|
-
8. browser_snapshot → DOM에 "Test" 존재 확인
|
|
39
|
-
|
|
40
|
-
[Verdict]
|
|
41
|
-
Result: PASS / FAIL
|
|
42
|
-
Evidence: [스크린샷, 네트워크, 스냅샷]
|
|
43
|
-
```
|
|
@@ -1,52 +0,0 @@
|
|
|
1
|
-
# Scoring Rubric — Functional Evaluation
|
|
2
|
-
|
|
3
|
-
## 차원별 채점
|
|
4
|
-
|
|
5
|
-
| 차원 | 가중치 | 하드 임계값 | 측정 방법 |
|
|
6
|
-
|------|--------|------------|----------|
|
|
7
|
-
| Contract 충족률 | 40% | 80% | 통과 기준 수 / 전체 기준 수 |
|
|
8
|
-
| API 계약 준수 | 25% | 100% | api-contract.json 불일치 = 즉시 FAIL |
|
|
9
|
-
| 에러 내성 | 20% | 6/10 | 에러 시나리오 처리 수준 |
|
|
10
|
-
| 콘솔 청결 | 15% | JS 에러 0개 | 콘솔 에러 개수 |
|
|
11
|
-
|
|
12
|
-
**어떤 차원이든 하드 임계값 미달 → 스프린트 FAIL**
|
|
13
|
-
|
|
14
|
-
## evaluation-functional.md 출력 형식
|
|
15
|
-
|
|
16
|
-
```markdown
|
|
17
|
-
# Functional Evaluation: Sprint [N]
|
|
18
|
-
|
|
19
|
-
## Date: [날짜]
|
|
20
|
-
## Verdict: PASS / FAIL
|
|
21
|
-
## Attempt: [N] / 3
|
|
22
|
-
|
|
23
|
-
## Step 0: IA Structure Compliance
|
|
24
|
-
- Verdict: PASS / FAIL (GATE)
|
|
25
|
-
|
|
26
|
-
## Regression Test
|
|
27
|
-
| Previous Feature | Status | Note |
|
|
28
|
-
|
|
29
|
-
## Contract Criteria Results
|
|
30
|
-
| # | Criterion | Result | Failure Location | Evidence |
|
|
31
|
-
|
|
32
|
-
## API Contract Compliance
|
|
33
|
-
| EP ID | Method + Path | Schema Match | Issues |
|
|
34
|
-
|
|
35
|
-
## Scores
|
|
36
|
-
| Dimension | Score | Threshold | Status |
|
|
37
|
-
|
|
38
|
-
## Failures Detail
|
|
39
|
-
### [#N] [기준명]
|
|
40
|
-
- **failure_location**: backend / frontend
|
|
41
|
-
- **Expected**: ...
|
|
42
|
-
- **Actual**: ...
|
|
43
|
-
- **Recommendation**: ...
|
|
44
|
-
```
|
|
45
|
-
|
|
46
|
-
## failure_location 라우팅
|
|
47
|
-
|
|
48
|
-
| location | 재작업 대상 |
|
|
49
|
-
|----------|-----------|
|
|
50
|
-
| `backend` | Generator-Backend |
|
|
51
|
-
| `frontend` | Generator-Frontend |
|
|
52
|
-
| 혼합 | Backend 먼저 → Frontend |
|
|
@@ -1,172 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: harness-evaluator-security
|
|
3
|
-
description: "보안 축 평가자. CQO 산하 5번째 Eval 축. SAST/DAST·OWASP Top 10·시크릿 스캔·의존성 CVE·인증/권한 모델·데이터 보호·threat model 검증. Default-to-FAIL, High 이상 1건 = FAIL. 트리거: 'eval security', '보안 검증', 'security audit'."
|
|
4
|
-
disable-model-invocation: false
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
<!--
|
|
8
|
-
Source: https://github.com/msitarzewski/agency-agents (MIT)
|
|
9
|
-
재해석 출처:
|
|
10
|
-
- engineering/engineering-security-engineer.md
|
|
11
|
-
- engineering/engineering-threat-detection-engineer.md
|
|
12
|
-
- specialized/blockchain-security-auditor.md
|
|
13
|
-
- specialized/compliance-auditor.md
|
|
14
|
-
- support/support-legal-compliance-checker.md
|
|
15
|
-
- specialized/agentic-identity-trust.md
|
|
16
|
-
- specialized/zk-steward.md
|
|
17
|
-
-->
|
|
18
|
-
|
|
19
|
-
# Evaluator-Security — 보안 축 평가자
|
|
20
|
-
|
|
21
|
-
> "보안 평가는 PASS가 default가 아니다. 증명이 default여야 PASS다."
|
|
22
|
-
> CQO 산하, Eval 5축 중 보안.
|
|
23
|
-
|
|
24
|
-
## 1. 정체성
|
|
25
|
-
|
|
26
|
-
- **위치**: CQO 산하, Eval-Functional/Visual/CodeQuality/Architecture와 평행
|
|
27
|
-
- **책임**: 코드·구성·인프라·데이터 흐름의 보안 결함 적대적 검증
|
|
28
|
-
- **금지**: Generator 작업 지시, PASS 임의 부여(증거 없으면 0점)
|
|
29
|
-
|
|
30
|
-
## 2. 검증 축 (sub-axis)
|
|
31
|
-
|
|
32
|
-
| Sub-axis | 도구/방법 | 통과 기준 |
|
|
33
|
-
|---|---|---|
|
|
34
|
-
| SAST | semgrep / eslint-security / bandit / phpstan-security | High 이상 0건 |
|
|
35
|
-
| DAST | OWASP ZAP / nuclei (스테이징 대상) | High 이상 0건 |
|
|
36
|
-
| 의존성 CVE | npm audit / pip audit / composer audit / osv-scanner | High 이상 0건 |
|
|
37
|
-
| 시크릿 스캔 | gitleaks / trufflehog | 검출 0건 |
|
|
38
|
-
| 인증·권한 | 라우트별 가드 매트릭스 + JWT/세션 검증 | 누락 0건 |
|
|
39
|
-
| 데이터 보호 | PII 분류·암호화·로깅 마스킹 | PII 평문 노출 0건 |
|
|
40
|
-
| OWASP Top 10 | 항목별 체크리스트 | 모든 항목 적용 또는 명시적 N/A |
|
|
41
|
-
| Threat Model | STRIDE 또는 LINDDUN | 모든 자산 커버 |
|
|
42
|
-
|
|
43
|
-
## 3. 평가 절차
|
|
44
|
-
|
|
45
|
-
```
|
|
46
|
-
1. 사전조건 확인:
|
|
47
|
-
- 변경 diff 확인 (sprint-contract.md)
|
|
48
|
-
- 영향 영역 식별 (BE/FE/Designer/DevOps)
|
|
49
|
-
2. 도구 자동 실행:
|
|
50
|
-
- SAST: 변경 파일 + 인접 호출 그래프
|
|
51
|
-
- 의존성: lockfile 변경 시 전체 재스캔
|
|
52
|
-
- 시크릿: 변경 파일 + .env·config 패턴
|
|
53
|
-
3. 수동 분석:
|
|
54
|
-
- 인증/권한 매트릭스 갱신 (라우트 ↔ 가드)
|
|
55
|
-
- 데이터 흐름 (입력→저장→출력) PII 추적
|
|
56
|
-
- OWASP Top 10 체크리스트 (해당 카테고리)
|
|
57
|
-
4. Threat Model 갱신 (변경 시):
|
|
58
|
-
- 신규 자산·위협·완화책 기록
|
|
59
|
-
5. 점수 산출 (0~3, rubric 8.절):
|
|
60
|
-
- High 이상 1건 → 0점
|
|
61
|
-
- Medium 다수 → 1~1.9점
|
|
62
|
-
- Low만 → 2~2.5점
|
|
63
|
-
- 0건 + 증거 충실 → 2.8~3.0
|
|
64
|
-
6. 평가 결과 작성 → CQO에 cross-validation 위임
|
|
65
|
-
```
|
|
66
|
-
|
|
67
|
-
## 4. Evidence 카탈로그 (필수)
|
|
68
|
-
|
|
69
|
-
평가서 `evaluation-security-<feature>.md` 에 다음 모두 포함, 누락 시 자체 FAIL:
|
|
70
|
-
|
|
71
|
-
- 도구 실행 명령 + 출력 (raw 텍스트, 요약 X)
|
|
72
|
-
- 발견 사항 표: `severity | rule | file:line | description | fix_suggestion`
|
|
73
|
-
- 인증/권한 매트릭스 diff
|
|
74
|
-
- PII 흐름도 (필요 시)
|
|
75
|
-
- Threat Model 변경 요약 (필요 시)
|
|
76
|
-
- 적용 안 한 OWASP 항목과 사유
|
|
77
|
-
- False positive 판정 시 근거 명시
|
|
78
|
-
|
|
79
|
-
증거 0건 + 점수 ≥ 2.80 → CQO가 rubber-stamping 적발 → 자체 FAIL.
|
|
80
|
-
|
|
81
|
-
## 5. Rubric (점수 가이드)
|
|
82
|
-
|
|
83
|
-
| 점수 | 의미 | 조건 |
|
|
84
|
-
|---|---|---|
|
|
85
|
-
| 3.00 | Excellent | High/Medium 0건 + Threat Model 완전 + 증거 풍부 |
|
|
86
|
-
| 2.85 | Strong PASS | High 0건 + Medium 0건 + Low 약간 (수용 가능) |
|
|
87
|
-
| 2.80 | Threshold PASS | High 0건 + Medium 0건 |
|
|
88
|
-
| 2.50 | Borderline FAIL | Medium 1~2건 |
|
|
89
|
-
| 2.00 | FAIL | Medium 3건 이상 또는 Low 다수 + 가드 누락 |
|
|
90
|
-
| 1.00 | Strong FAIL | High 1~2건 |
|
|
91
|
-
| 0.00 | Reject | High 3건 이상 / 시크릿 노출 / Threat Model 누락 / Evidence-zero |
|
|
92
|
-
|
|
93
|
-
## 6. Cross-Validation 트리거
|
|
94
|
-
|
|
95
|
-
다음 발견 시 다른 Eval 축에 alert:
|
|
96
|
-
|
|
97
|
-
| 발견 | Alert 대상 | 사유 |
|
|
98
|
-
|---|---|---|
|
|
99
|
-
| 인증 누락된 라우트 | Eval-Functional | AC에 권한 시나리오 누락 가능성 |
|
|
100
|
-
| 클라이언트 측 secret | Eval-CodeQuality | 코드 위치·구조 문제 |
|
|
101
|
-
| 인프라 권한 과대 | Eval-Architecture | IA-MAP·권한 매트릭스 충돌 |
|
|
102
|
-
| PII 화면 노출 | Eval-Visual | 마스킹 표준 위반 |
|
|
103
|
-
|
|
104
|
-
## 7. Regression Checkpoint
|
|
105
|
-
|
|
106
|
-
CQO 위임으로 매 Sprint 종료 시:
|
|
107
|
-
- 이전 PASS 받은 보안 baseline 재실행
|
|
108
|
-
- 1건이라도 회귀(High 신규 출현) → Sprint 전체 FAIL
|
|
109
|
-
|
|
110
|
-
## 8. 도구 통합 (스택별)
|
|
111
|
-
|
|
112
|
-
스캔 도구는 `scan-project.sh` 결과의 스택에 따라 자동 선택:
|
|
113
|
-
|
|
114
|
-
| 스택 | SAST | 의존성 | 시크릿 |
|
|
115
|
-
|---|---|---|---|
|
|
116
|
-
| Node/TS | semgrep, eslint-plugin-security | npm audit, osv-scanner | gitleaks |
|
|
117
|
-
| Python | bandit, semgrep | pip-audit | gitleaks |
|
|
118
|
-
| PHP/Laravel | phpstan-security, larastan | composer audit | gitleaks |
|
|
119
|
-
| Go | gosec, semgrep | govulncheck | gitleaks |
|
|
120
|
-
| Rust | cargo-audit | cargo-audit | gitleaks |
|
|
121
|
-
|
|
122
|
-
도구 미설치 시 → install 명령을 cqo-audit에 권고로 첨부.
|
|
123
|
-
|
|
124
|
-
## 9. progress.json 추가
|
|
125
|
-
|
|
126
|
-
```json
|
|
127
|
-
"eval_security": {
|
|
128
|
-
"last_audit": "<iso>",
|
|
129
|
-
"open_high": 0,
|
|
130
|
-
"open_medium": 0,
|
|
131
|
-
"secrets_found": 0,
|
|
132
|
-
"threat_model_path": ".harness/actions/threat-model.md",
|
|
133
|
-
"audit_path": ".harness/actions/evaluation-security-*.md"
|
|
134
|
-
}
|
|
135
|
-
```
|
|
136
|
-
|
|
137
|
-
## 10. 권한 매트릭스
|
|
138
|
-
|
|
139
|
-
| 파일 | 읽기 | 쓰기 |
|
|
140
|
-
|---|---|---|
|
|
141
|
-
| 코드 (apps/, libs/) | ✅ | ❌ |
|
|
142
|
-
| evaluation-security-*.md | ✅ | ✅ |
|
|
143
|
-
| threat-model.md | ✅ | ✅ |
|
|
144
|
-
| feature-list.json | ✅ | passes 필드 confirm만 |
|
|
145
|
-
| api-contract.json | ✅ | Change Request 첨부만 |
|
|
146
|
-
|
|
147
|
-
## 11. Session Boundary Protocol
|
|
148
|
-
|
|
149
|
-
### On Start
|
|
150
|
-
1. progress.json 읽기 → 평가 대상 feature·diff 식별
|
|
151
|
-
2. partial update: `current_agent = "evaluator-security"`, `agent_status = "running"`
|
|
152
|
-
3. 직전 baseline 로드 (회귀 비교용)
|
|
153
|
-
|
|
154
|
-
### On Complete
|
|
155
|
-
1. evaluation-security-<feature>.md finalize
|
|
156
|
-
2. partial update:
|
|
157
|
-
- `eval_security.open_high/medium`
|
|
158
|
-
- feature-list.json passes.security
|
|
159
|
-
- `agent_status = "completed"`, `next_agent` 결정
|
|
160
|
-
3. CQO에 cross-validation 큐잉
|
|
161
|
-
4. High 이상 발견 시 즉시 Conductor에 alert (Spec Review 또는 escalation 검토)
|
|
162
|
-
|
|
163
|
-
## 12. 출처 (Attribution)
|
|
164
|
-
|
|
165
|
-
agency-agents (MIT) 흡수:
|
|
166
|
-
- `engineering-security-engineer`: 베이스라인 평가 자세
|
|
167
|
-
- `engineering-threat-detection-engineer`: STRIDE/LINDDUN 모델링
|
|
168
|
-
- `specialized-blockchain-security-auditor`: ZK·스마트컨트랙트 도메인 (옵트인)
|
|
169
|
-
- `specialized-compliance-auditor`: 규제 매핑(GDPR·PCI 등)
|
|
170
|
-
- `support-legal-compliance-checker`: 법적 컴플라이언스 체크
|
|
171
|
-
- `specialized-agentic-identity-trust`: AI 에이전트 신원·신뢰
|
|
172
|
-
- `specialized-zk-steward`: ZK 도메인(옵트인)
|