bros-harness 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +7 -0
- package/LICENSE +21 -0
- package/README.md +183 -0
- package/SECURITY.md +16 -0
- package/assets/agents.manifest.json +55 -0
- package/assets/commands.manifest.json +35 -0
- package/assets/docs.manifest.json +20 -0
- package/assets/import-report.md +25 -0
- package/assets/manifest.json +799 -0
- package/assets/opencode/agents/README.md +3 -0
- package/assets/opencode/agents/bro-build.md +256 -0
- package/assets/opencode/agents/bro-design.md +77 -0
- package/assets/opencode/agents/bro-docs.md +72 -0
- package/assets/opencode/agents/bro-explore.md +143 -0
- package/assets/opencode/agents/bro-ops.md +195 -0
- package/assets/opencode/agents/bro-shield.md +77 -0
- package/assets/opencode/agents/bro-test.md +204 -0
- package/assets/opencode/agents/bro-ui.md +135 -0
- package/assets/opencode/agents/mighty-bro.md +252 -0
- package/assets/opencode/commands/README.md +3 -0
- package/assets/opencode/commands/bros-assemble.md +32 -0
- package/assets/opencode/commands/bros-build.md +58 -0
- package/assets/opencode/commands/bros-plan.md +83 -0
- package/assets/opencode/commands/bros-review.md +38 -0
- package/assets/opencode/commands/bros-status.md +26 -0
- package/assets/opencode/docs/README.md +3 -0
- package/assets/opencode/docs/bros-builtin-skills.md +63 -0
- package/assets/opencode/docs/bros-harness.md +194 -0
- package/assets/opencode/skills/README.md +3 -0
- package/assets/opencode/skills/agent-architecture-audit/SKILL.md +256 -0
- package/assets/opencode/skills/agent-harness-construction/.openskills.json +7 -0
- package/assets/opencode/skills/agent-harness-construction/SKILL.md +73 -0
- package/assets/opencode/skills/agent-introspection-debugging/.openskills.json +7 -0
- package/assets/opencode/skills/agent-introspection-debugging/SKILL.md +153 -0
- package/assets/opencode/skills/api-design/.openskills.json +7 -0
- package/assets/opencode/skills/api-design/agents/openai.yaml +7 -0
- package/assets/opencode/skills/architecture-decision-records/.openskills.json +7 -0
- package/assets/opencode/skills/architecture-decision-records/SKILL.md +179 -0
- package/assets/opencode/skills/article-writing/.openskills.json +7 -0
- package/assets/opencode/skills/article-writing/SKILL.md +79 -0
- package/assets/opencode/skills/article-writing/agents/openai.yaml +7 -0
- package/assets/opencode/skills/automation-audit-ops/.openskills.json +7 -0
- package/assets/opencode/skills/automation-audit-ops/SKILL.md +142 -0
- package/assets/opencode/skills/backend-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/backend-patterns/SKILL.md +561 -0
- package/assets/opencode/skills/backend-patterns/agents/openai.yaml +7 -0
- package/assets/opencode/skills/benchmark/.openskills.json +7 -0
- package/assets/opencode/skills/benchmark/SKILL.md +93 -0
- package/assets/opencode/skills/bros-orchestrate/SKILL.md +455 -0
- package/assets/opencode/skills/browser-qa/.openskills.json +7 -0
- package/assets/opencode/skills/browser-qa/SKILL.md +87 -0
- package/assets/opencode/skills/canary-watch/.openskills.json +7 -0
- package/assets/opencode/skills/canary-watch/SKILL.md +107 -0
- package/assets/opencode/skills/code-review-expert/SKILL.md +155 -0
- package/assets/opencode/skills/code-review-expert/agents/agent.yaml +7 -0
- package/assets/opencode/skills/code-review-expert/references/code-quality-checklist.md +130 -0
- package/assets/opencode/skills/code-review-expert/references/removal-plan.md +52 -0
- package/assets/opencode/skills/code-review-expert/references/security-checklist.md +118 -0
- package/assets/opencode/skills/code-review-expert/references/solid-checklist.md +65 -0
- package/assets/opencode/skills/code-tour/.openskills.json +7 -0
- package/assets/opencode/skills/code-tour/SKILL.md +236 -0
- package/assets/opencode/skills/coding-standards/.openskills.json +7 -0
- package/assets/opencode/skills/coding-standards/SKILL.md +549 -0
- package/assets/opencode/skills/coding-standards/agents/openai.yaml +7 -0
- package/assets/opencode/skills/context-budget/.openskills.json +7 -0
- package/assets/opencode/skills/context-budget/SKILL.md +135 -0
- package/assets/opencode/skills/database-migrations/.openskills.json +7 -0
- package/assets/opencode/skills/database-migrations/SKILL.md +429 -0
- package/assets/opencode/skills/deployment-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/deployment-patterns/SKILL.md +427 -0
- package/assets/opencode/skills/design-system/.openskills.json +7 -0
- package/assets/opencode/skills/design-system/SKILL.md +82 -0
- package/assets/opencode/skills/docker-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/docker-patterns/SKILL.md +364 -0
- package/assets/opencode/skills/documentation-lookup/.openskills.json +7 -0
- package/assets/opencode/skills/documentation-lookup/SKILL.md +90 -0
- package/assets/opencode/skills/documentation-lookup/agents/openai.yaml +7 -0
- package/assets/opencode/skills/e2e-testing/.openskills.json +7 -0
- package/assets/opencode/skills/e2e-testing/SKILL.md +326 -0
- package/assets/opencode/skills/e2e-testing/agents/openai.yaml +7 -0
- package/assets/opencode/skills/error-handling/SKILL.md +376 -0
- package/assets/opencode/skills/frontend-design/.openskills.json +7 -0
- package/assets/opencode/skills/frontend-design/SKILL.md +145 -0
- package/assets/opencode/skills/frontend-design-direction/SKILL.md +92 -0
- package/assets/opencode/skills/frontend-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/frontend-patterns/SKILL.md +642 -0
- package/assets/opencode/skills/frontend-patterns/agents/openai.yaml +7 -0
- package/assets/opencode/skills/gateguard/.openskills.json +7 -0
- package/assets/opencode/skills/gateguard/SKILL.md +125 -0
- package/assets/opencode/skills/git-master/SKILL.md +60 -0
- package/assets/opencode/skills/golang-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/golang-patterns/SKILL.md +674 -0
- package/assets/opencode/skills/golang-testing/.openskills.json +7 -0
- package/assets/opencode/skills/golang-testing/SKILL.md +720 -0
- package/assets/opencode/skills/grafana-dashboard-design/SKILL.md +65 -0
- package/assets/opencode/skills/hexagonal-architecture/.openskills.json +7 -0
- package/assets/opencode/skills/hexagonal-architecture/SKILL.md +276 -0
- package/assets/opencode/skills/java-coding-standards/.openskills.json +7 -0
- package/assets/opencode/skills/java-coding-standards/SKILL.md +383 -0
- package/assets/opencode/skills/jpa-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/jpa-patterns/SKILL.md +151 -0
- package/assets/opencode/skills/knowledge-ops/.openskills.json +7 -0
- package/assets/opencode/skills/knowledge-ops/SKILL.md +154 -0
- package/assets/opencode/skills/make-interfaces-feel-better/SKILL.md +151 -0
- package/assets/opencode/skills/mysql-patterns/SKILL.md +412 -0
- package/assets/opencode/skills/nestjs-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/nestjs-patterns/SKILL.md +230 -0
- package/assets/opencode/skills/nextjs-turbopack/.openskills.json +7 -0
- package/assets/opencode/skills/nextjs-turbopack/SKILL.md +57 -0
- package/assets/opencode/skills/nextjs-turbopack/agents/openai.yaml +7 -0
- package/assets/opencode/skills/parallel-execution-optimizer/SKILL.md +72 -0
- package/assets/opencode/skills/postgres-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/postgres-patterns/SKILL.md +147 -0
- package/assets/opencode/skills/prisma-patterns/SKILL.md +371 -0
- package/assets/opencode/skills/product-capability/.openskills.json +7 -0
- package/assets/opencode/skills/product-capability/SKILL.md +141 -0
- package/assets/opencode/skills/product-lens/.openskills.json +7 -0
- package/assets/opencode/skills/product-lens/SKILL.md +92 -0
- package/assets/opencode/skills/production-audit/SKILL.md +206 -0
- package/assets/opencode/skills/python-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/python-patterns/SKILL.md +750 -0
- package/assets/opencode/skills/python-testing/.openskills.json +7 -0
- package/assets/opencode/skills/python-testing/SKILL.md +816 -0
- package/assets/opencode/skills/redis-patterns/SKILL.md +403 -0
- package/assets/opencode/skills/requirements-clarity/README.md +260 -0
- package/assets/opencode/skills/requirements-clarity/SKILL.md +324 -0
- package/assets/opencode/skills/rust-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/rust-patterns/SKILL.md +499 -0
- package/assets/opencode/skills/rust-testing/.openskills.json +7 -0
- package/assets/opencode/skills/rust-testing/SKILL.md +500 -0
- package/assets/opencode/skills/safety-guard/.openskills.json +7 -0
- package/assets/opencode/skills/safety-guard/SKILL.md +75 -0
- package/assets/opencode/skills/search-first/.openskills.json +7 -0
- package/assets/opencode/skills/search-first/SKILL.md +181 -0
- package/assets/opencode/skills/security-review/.openskills.json +7 -0
- package/assets/opencode/skills/security-review/agents/openai.yaml +7 -0
- package/assets/opencode/skills/security-review/cloud-infrastructure-security.md +361 -0
- package/assets/opencode/skills/security-scan/.openskills.json +7 -0
- package/assets/opencode/skills/security-scan/SKILL.md +165 -0
- package/assets/opencode/skills/springboot-patterns/.openskills.json +7 -0
- package/assets/opencode/skills/springboot-patterns/SKILL.md +314 -0
- package/assets/opencode/skills/springboot-tdd/.openskills.json +7 -0
- package/assets/opencode/skills/springboot-tdd/SKILL.md +158 -0
- package/assets/opencode/skills/springboot-verification/.openskills.json +7 -0
- package/assets/opencode/skills/springboot-verification/SKILL.md +231 -0
- package/assets/opencode/skills/strategic-compact/.openskills.json +7 -0
- package/assets/opencode/skills/strategic-compact/SKILL.md +131 -0
- package/assets/opencode/skills/strategic-compact/agents/openai.yaml +7 -0
- package/assets/opencode/skills/strategic-compact/suggest-compact.sh +54 -0
- package/assets/opencode/skills/tdd-workflow/.openskills.json +7 -0
- package/assets/opencode/skills/tdd-workflow/SKILL.md +463 -0
- package/assets/opencode/skills/tdd-workflow/agents/openai.yaml +7 -0
- package/assets/opencode/skills/verification-loop/.openskills.json +7 -0
- package/assets/opencode/skills/verification-loop/SKILL.md +126 -0
- package/assets/opencode/skills/verification-loop/agents/openai.yaml +7 -0
- package/assets/opencode/skills/vite-patterns/SKILL.md +449 -0
- package/assets/opencode/skills/web-doc-search/SKILL.md +51 -0
- package/assets/opencode/templates/README.md +3 -0
- package/assets/opencode/templates/bros/adr.md +39 -0
- package/assets/opencode/templates/bros/delivery-report.md +71 -0
- package/assets/opencode/templates/bros/explorer-evidence-packet.md +51 -0
- package/assets/opencode/templates/bros/prd.md +72 -0
- package/assets/opencode/templates/bros/security-review.md +48 -0
- package/assets/opencode/templates/bros/status-board.md +33 -0
- package/assets/opencode/templates/bros/task-packet.md +94 -0
- package/assets/opencode/templates/bros/test-strategy.md +57 -0
- package/assets/opencode/templates/bros/ui-implementation-packet.md +64 -0
- package/assets/skills.manifest.json +650 -0
- package/assets/templates.manifest.json +55 -0
- package/bin/bros.mjs +122 -0
- package/docs/compatibility.md +9 -0
- package/docs/installation.md +66 -0
- package/docs/integrations/claude.md +5 -0
- package/docs/integrations/codex.md +5 -0
- package/docs/integrations/opencode.md +39 -0
- package/docs/migration/from-local-opencode-config.md +10 -0
- package/docs/release-process.md +11 -0
- package/docs/repository-structure.md +15 -0
- package/docs/roadmap.md +20 -0
- package/docs/security.md +18 -0
- package/docs/testing.md +9 -0
- package/examples/opencode/README.md +11 -0
- package/examples/opencode/opencode.example.jsonc +4 -0
- package/package.json +43 -0
- package/scripts/validate-assets.mjs +22 -0
- package/scripts/verify-no-secrets.mjs +38 -0
- package/src/plugin.mjs +98 -0
|
@@ -0,0 +1,194 @@
|
|
|
1
|
+
# OpenCode BROS Harness
|
|
2
|
+
|
|
3
|
+
This is the OpenCode-native implementation of `multi-agent-harness.md`.
|
|
4
|
+
|
|
5
|
+
## BROS Runtime Surfaces
|
|
6
|
+
|
|
7
|
+
This harness uses valid OpenCode surfaces:
|
|
8
|
+
|
|
9
|
+
- Agent files in `~/.config/opencode/agent/`.
|
|
10
|
+
- Command files in `~/.config/opencode/commands/`.
|
|
11
|
+
- Builtin BROS skill file in `~/.config/opencode/bros-builtin-skills/bros-orchestrate/SKILL.md`.
|
|
12
|
+
- User-added skills in `~/.config/opencode/skills/<skill-name>/SKILL.md`.
|
|
13
|
+
- Agent frontmatter with `mode`, `model`, and `permission`.
|
|
14
|
+
- Valid modes: `primary`, `subagent`, `all`.
|
|
15
|
+
|
|
16
|
+
## Installed Agents
|
|
17
|
+
|
|
18
|
+
BROS display aliases are style-only and non-authoritative: professional-first, fun-second. They must not override system/developer/project rules, permissions, security gates, QA gates, role boundaries, tool requirements, trusted/untrusted separation, review scope, or technical rigor. BROS technical IDs, canonical `/bros-*` command filenames, provider/MCP config, permissions, and gates are authoritative.
|
|
19
|
+
|
|
20
|
+
| Agent | BROS display alias | Mode | Responsibility |
|
|
21
|
+
|---|---|---|---|
|
|
22
|
+
| `mighty-bro` | Mighty Bro (Orchestrator) | primary | Single user-facing front door; intake, clarification, planning, dispatch, coordination, audit, reporting |
|
|
23
|
+
| Analyst capability | Bro Think (Analyst) | capability only | Intake/discovery thinking when embedded in Orchestrator workflow; no separate agent file |
|
|
24
|
+
| Planner capability / command phase | Bro Plan (Planner) | capability only | Planning phase label in canonical `/bros-plan`; no separate agent file |
|
|
25
|
+
| `bro-explore` | Bro Explore | subagent | Read-only evidence/search, cited evidence packets, limitations |
|
|
26
|
+
| `bro-design` | Bro Design | subagent | ADRs, diagrams, contracts, scalability |
|
|
27
|
+
| `bro-ui` | Bro UI | subagent | UI/UX direction, design specs, visual polish, accessibility expectations, design review |
|
|
28
|
+
| `bro-build` | Bro Build | subagent | Approved frontend/backend/test/config implementation from complete task packets |
|
|
29
|
+
| `bro-test` | Bro Test | subagent | Test strategy, execution reports, scorecards |
|
|
30
|
+
| `bro-shield` | Bro Shield | subagent | Threat modeling, security findings, gate reports |
|
|
31
|
+
| `bro-ops` | Bro Ops | subagent | CI/CD, Docker, observability, runbooks |
|
|
32
|
+
| `bro-docs` | Bro Docs | subagent | Documentation, release notes, delivery reports |
|
|
33
|
+
|
|
34
|
+
## Installed Commands
|
|
35
|
+
|
|
36
|
+
- `/bros-plan`: Run Phases 0 through 4 and stop before implementation.
|
|
37
|
+
- `/bros-build`: Execute an approved plan through implementation, quality, security, docs, and delivery.
|
|
38
|
+
- `/bros-assemble`: Run end-to-end planning plus build through final delivery while preserving all security, destructive-operation, production/cloud, secret, QA, architecture, and governance gates.
|
|
39
|
+
- `/bros-status`: Summarize phase, gate, blocker, and artifact state.
|
|
40
|
+
- `/bros-review`: Audit a plan or delivery for role, gate, and native-config compliance.
|
|
41
|
+
|
|
42
|
+
BROS commands run inline in the current conversation. They do not spawn nested `mighty-bro` sessions.
|
|
43
|
+
|
|
44
|
+
## Builtin Skill Pack
|
|
45
|
+
|
|
46
|
+
The BROS agents have a curated builtin skill pack installed under `~/.config/opencode/bros-builtin-skills`. See the BROS builtin skills reference for the routing map.
|
|
47
|
+
|
|
48
|
+
Additional skill folders added to `~/.config/opencode/skills/<skill-name>/SKILL.md` are kept separate from the builtin pack and are available to agents when approved and relevant.
|
|
49
|
+
|
|
50
|
+
## Workflow
|
|
51
|
+
|
|
52
|
+
1. Run `/bros-plan "<request>"`.
|
|
53
|
+
2. The Orchestrator performs intake, classifies scope/depth/risk, asks clarification when risk/scope is unclear, or states assumptions and proceeds.
|
|
54
|
+
3. Review the plan, acceptance criteria, architecture, technical review, and task packets.
|
|
55
|
+
4. Approve or request changes.
|
|
56
|
+
5. Run `/bros-build "<approved plan path or pasted plan>"`.
|
|
57
|
+
6. Use `/bros-status` or `/bros-review` as needed.
|
|
58
|
+
|
|
59
|
+
## Orchestrator-First Orchestration
|
|
60
|
+
|
|
61
|
+
The CTO Orchestrator (`mighty-bro`, Mighty Bro) coordinates rather than executes production work. It is the single user-facing front door for canonical `/bros-plan`, `/bros-build`, intake, clarification, scope/depth/risk classification, planning, dispatch, coordination, audit, and reporting.
|
|
62
|
+
|
|
63
|
+
The Orchestrator owns embedded PM/discovery/planning capability for BROS intake, including discovery notes, scope statements, user stories, acceptance criteria, NFRs, and task packets when risk and scope are clear enough.
|
|
64
|
+
|
|
65
|
+
Before planning or dispatch, the Orchestrator prepares a visible Orchestrator Intake Brief with trusted policy/gates, untrusted user request, restatement, desired outcome, context, classification, assumptions, ambiguities/risks, investigation paths, out-of-scope items, expected deliverable, and optional specialist dispatch.
|
|
66
|
+
|
|
67
|
+
Handling rules:
|
|
68
|
+
|
|
69
|
+
- Simple: answer inline or produce concise plan/status.
|
|
70
|
+
- Ambiguous: ask targeted clarification when risk/scope is unclear; otherwise state assumptions and proceed.
|
|
71
|
+
- Evidence-needed: dispatch `bro-explore` for read-only citations before planning or implementation decisions.
|
|
72
|
+
- UI/design: dispatch `bro-ui` for design direction, specification, accessibility expectations, or design review.
|
|
73
|
+
- Small: use minimal/no specialists and skip Architect with rationale when localized and low-risk.
|
|
74
|
+
- Medium implementation: validate/create bounded Phase 0-4 planning and dispatch `bro-build` from approved task packets; Architect may be skipped when coupling is low.
|
|
75
|
+
- Complex/implementation: require `bro-design`, run Phase 0-4 planning, then use canonical `/bros-build` to dispatch `bro-build` and review/doc agents for approved task packets.
|
|
76
|
+
- Security-sensitive: trigger `bro-shield`; stop on missing approvals, CRITICAL findings, destructive actions, or unclear production risk.
|
|
77
|
+
|
|
78
|
+
Every routed workflow must emit a routing record with classification, selected agents, skipped agents rationale, gates, and stop conditions.
|
|
79
|
+
|
|
80
|
+
For normal prompts that are only exploratory, diagnostic, or clarification-oriented, the active agent should answer in the current conversation. It should not spawn `mighty-bro` or nested subtask chains.
|
|
81
|
+
|
|
82
|
+
## Rendering
|
|
83
|
+
|
|
84
|
+
Do not show patch transcripts or deleted lines with Markdown strikethrough. Use normal summaries, and only use fenced `diff` or `text` blocks when a patch excerpt is explicitly needed.
|
|
85
|
+
|
|
86
|
+
## Investigation Permissions
|
|
87
|
+
|
|
88
|
+
BROS agents can read, glob, grep, and use configured builtin or user-added skills without repeated approval prompts. Shell and write access remain role-restricted.
|
|
89
|
+
|
|
90
|
+
## Local Command Permission Model
|
|
91
|
+
|
|
92
|
+
OpenCode BROS agents use pattern-based Bash permissions, not broad `bash: allow`. Public OSS defaults keep dependency installation, arbitrary package scripts, Docker runtime/mutation, destructive operations, cloud/production mutation, and secret-reading approval-gated.
|
|
93
|
+
|
|
94
|
+
- `bro-build`: may run local project inspection, git read-only inspection, dependency-free test/build/lint/typecheck commands, and localhost curl checks; edits, installs, arbitrary package scripts, Docker runtime/mutation, and high-risk operations remain approval-gated by task packet scope.
|
|
95
|
+
- `bro-test`: edit remains denied; local inspection, git read-only inspection, dependency-free test/lint/typecheck/build checks, Playwright tests, and localhost curl commands are allowlisted for QA evidence; installs and Docker runtime/mutation require approval.
|
|
96
|
+
- `bro-ops`: may run local inspection, git read-only inspection, dependency-free verification commands, Playwright tests, and localhost curl; Docker runtime/mutation and edits to OpenCode config remain approval-gated or denied according to role policy.
|
|
97
|
+
- `bro-explore`: may run read-only inspection Bash only (`pwd`, `ls*`, `find*`, `tree*`, `rg*`, `grep*`, read-only git status/diff/log, `cat *`, `sed -n*`, `head*`, `tail*`, `wc*`) with edits, installs, Docker runtime, writes, and destructive commands denied.
|
|
98
|
+
- `mighty-bro`: bash and edit remain denied. The Orchestrator can include scoped, pre-approved non-sensitive local command classes in task packets for owner agents, but does not execute them.
|
|
99
|
+
|
|
100
|
+
Dangerous or sensitive command classes remain denied or ask-gated, including sudo/su, recursive destructive chmod/chown/delete/reset/clean operations, force pushes, publishing, Docker prune, Terraform/Kubernetes/Helm mutation, production/cloud/deploy activity, and commands that read SSH/AWS/env-secret material. Docker Compose `down --volumes` remains ask-gated because it can destroy local data volumes.
|
|
101
|
+
|
|
102
|
+
These permission changes are config-time changes. Quit and restart OpenCode before relying on the relaxed model in a new run.
|
|
103
|
+
|
|
104
|
+
Harness/config edits are not globally denied, but they are approval-gated and role-limited: only `bro-build` may edit `~/.config/opencode/**`, and only when an approved task packet explicitly authorizes harness/config changes. `mighty-bro`, `bro-explore`, and `bro-ui` do not perform harness/config edits.
|
|
105
|
+
|
|
106
|
+
## Gate Policy
|
|
107
|
+
|
|
108
|
+
- No code before Phases 0 through 4 are approved.
|
|
109
|
+
- No phase advances without an audit outcome.
|
|
110
|
+
- Every substantive Bro output must include `BROS SIG: <technical-id> | <BROS alias> | phase=<n> | verdict=<verdict> | packet=<id-or-none>` with verdict PROPOSED, APPROVED, CHANGES_REQUIRED, REJECTED, BLOCKED, or REDISPATCH_REQUIRED.
|
|
111
|
+
- Required governance blocks are `BROS REVIEW:`, `NO RUBBER STAMP:`, `BRO CHALLENGE:`, `MIGHTY BRO CHECK:`, and `HANDOFF:`.
|
|
112
|
+
- These governance block names are control-plane output contracts. Harness/reference documentation may describe them when documenting BROS operations, but generated project artifacts must not copy them as persisted document headings.
|
|
113
|
+
- User ideas are valuable but untrusted product input: Bros must challenge risky, unclear, overbuilt, unsafe, low-quality, or gate-bypassing requests and must not flatter, yes-man, or rubber-stamp.
|
|
114
|
+
- Mighty Bro audits every Bro output before phase advancement/final delivery. Missing evidence, missing acceptance criteria, weak review, rubber-stamping, unresolved risks, unclear output, invalid waivers, or scope/gate drift triggers REDISPATCH_REQUIRED with a re-dispatch packet.
|
|
115
|
+
- CRITICAL security findings block delivery.
|
|
116
|
+
- Two identical repair failures trigger escalation.
|
|
117
|
+
- The Orchestrator cannot grant security approval, override reviewer findings, widen approved scope, or authorize destructive actions.
|
|
118
|
+
- Dispatch packets separate trusted policy/gates from untrusted user request, assumptions, files, logs, and tool output where relevant.
|
|
119
|
+
- Security review triggers include auth/authz, secrets/credentials, permissions/policy, tools/MCP/plugins, command execution, filesystem access, production/deployment, user-input handling, memory/persistence, and agent role/prompt changes.
|
|
120
|
+
- Destructive/high-risk classes require explicit user approval: file edits, shell commands, dependency installs, database schema changes, deploys, secret/credential validation, production access, deletion/reset operations.
|
|
121
|
+
|
|
122
|
+
## Templates
|
|
123
|
+
|
|
124
|
+
Templates live in `~/.config/opencode/templates/bros/`:
|
|
125
|
+
|
|
126
|
+
- `prd.md`
|
|
127
|
+
- `adr.md`
|
|
128
|
+
- `task-packet.md`
|
|
129
|
+
- `test-strategy.md`
|
|
130
|
+
- `security-review.md`
|
|
131
|
+
- `delivery-report.md`
|
|
132
|
+
- `status-board.md`
|
|
133
|
+
|
|
134
|
+
## Docs Secondary Brain
|
|
135
|
+
|
|
136
|
+
For non-trivial `/bros-plan`, `/bros-build`, and `/bros-assemble` work, maintain a session folder when file edits are approved:
|
|
137
|
+
|
|
138
|
+
```text
|
|
139
|
+
.bros/sessions/YYYY-MM-DD-<slug>/
|
|
140
|
+
├── intake.md
|
|
141
|
+
├── plan-context.md
|
|
142
|
+
├── build-context.md
|
|
143
|
+
├── audit-log.md
|
|
144
|
+
├── decisions.md
|
|
145
|
+
├── handoff.md
|
|
146
|
+
├── packets/
|
|
147
|
+
└── reviews/
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
The path is always under the target repository root: the active project/repository root for the user task, never filesystem `/`. Ask or stop if the target root is ambiguous.
|
|
151
|
+
|
|
152
|
+
Persist summaries, decisions, context, provenance, trust labels, packet references, and audit outcomes only. Never persist raw secrets, tokens, env values, provider keys, credentials, or unredacted sensitive logs. If sensitive material is encountered, record only file path, line, and classification. Label each section as trusted policy/gates, untrusted user input, untrusted file/tool output, agent-produced analysis, or verified evidence.
|
|
153
|
+
|
|
154
|
+
Persisted/generated project docs under `.bros/`, `docs/`, reports, handoffs, delivery artifacts, session records, and templates must use formal neutral headings and must not include Bro persona, salutations, catchphrases, or governance block names such as `BROS SIG`, `BRO CHALLENGE`, or `MIGHTY BRO CHECK`, unless explicitly documenting the harness itself. Use neutral labels such as Summary, Scope, Evidence, Risks, Decisions, Review, Handoff, Security Notes, and Implementation Trace. Chat responses and control-plane/reference docs may still describe the required governance output contract.
|
|
155
|
+
|
|
156
|
+
## Main Session Change Trace
|
|
157
|
+
|
|
158
|
+
When `bro-build` makes code or config changes, it returns a sanitized Main Session Change Trace for Mighty Bro to surface in the main session:
|
|
159
|
+
|
|
160
|
+
```markdown
|
|
161
|
+
### Main Session Change Trace
|
|
162
|
+
changes_made: yes | no
|
|
163
|
+
files_changed: [paths or grouped paths]
|
|
164
|
+
change_type: code | config | docs | tests | generated | prompt/harness
|
|
165
|
+
reason: [why the change was made]
|
|
166
|
+
verification: [checks run or not run, with reason]
|
|
167
|
+
risks/follow-ups: [remaining risks or next steps]
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
Forbidden in the trace: raw secrets, env values, credentials, full raw diffs, unredacted logs, and large generated/vendor dumps. Patch excerpts are allowed only when explicitly requested and redacted.
|
|
171
|
+
|
|
172
|
+
Dual-label owner example for packets and status boards: `bro-build` (Bro Build), `bro-shield` (Bro Shield), or `mighty-bro` (Mighty Bro).
|
|
173
|
+
|
|
174
|
+
## Security Note
|
|
175
|
+
|
|
176
|
+
Provider credentials should be configured through your shell environment or another non-committed secret source. A known pre-existing provider credential issue in the current global OpenCode config is tracked separately; do not print, validate, rotate, or modify those values as part of BROS harness changes.
|
|
177
|
+
|
|
178
|
+
## Optional ECC References
|
|
179
|
+
|
|
180
|
+
ECC skill material may be consulted from `sanitized backup reference` as static reference material or to restore specific skills into either `bros-builtin-skills/` or the user-added `skills/` directory. Do not reinstall plugins, MCP servers, Node packages, routing files, or vendor dependencies unless you intentionally leave BROS-only mode.
|
|
181
|
+
|
|
182
|
+
For strict BROS-only startup, launch OpenCode with external skill loading disabled:
|
|
183
|
+
|
|
184
|
+
```bash
|
|
185
|
+
OPENCODE_DISABLE_EXTERNAL_SKILLS=1 OPENCODE_DISABLE_CLAUDE_CODE_SKILLS=1 opencode
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
## Restart Required
|
|
189
|
+
|
|
190
|
+
OpenCode loads config-time files at startup. Quit and restart OpenCode after installing or editing agents, commands, or skills.
|
|
191
|
+
|
|
192
|
+
## Canonical Routing
|
|
193
|
+
|
|
194
|
+
Canonical routing uses BROS IDs and `/bros-*` commands.
|
|
@@ -0,0 +1,256 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: agent-architecture-audit
|
|
3
|
+
description: Full-stack diagnostic for agent and LLM applications. Audits the 12-layer agent stack for wrapper regression, memory pollution, tool discipline failures, hidden repair loops, and rendering corruption. Produces severity-ranked findings with code-first fixes. Essential for developers building agent applications, autonomous loops, or any LLM-powered feature.
|
|
4
|
+
origin: oh-my-agent-check
|
|
5
|
+
tools: Read, Write, Edit, Bash, Grep, Glob
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Agent Architecture Audit
|
|
9
|
+
|
|
10
|
+
A diagnostic workflow for agent systems that hide failures behind wrapper layers, stale memory, retry loops, or transport/rendering mutations.
|
|
11
|
+
|
|
12
|
+
## When to Activate
|
|
13
|
+
|
|
14
|
+
**MANDATORY for:**
|
|
15
|
+
- Releasing any agent or LLM-powered application to production
|
|
16
|
+
- Shipping features with tool calling, memory, or multi-step workflows
|
|
17
|
+
- Agent behavior degrades after adding wrapper layers
|
|
18
|
+
- User reports "the agent is getting worse" or "tools are flaky"
|
|
19
|
+
- Same model works in playground but breaks inside your wrapper
|
|
20
|
+
- Debugging agent behavior for more than 15 minutes without finding root cause
|
|
21
|
+
|
|
22
|
+
**Especially critical when:**
|
|
23
|
+
- You've added new prompt layers, tool definitions, or memory systems
|
|
24
|
+
- Different agents in your system behave inconsistently
|
|
25
|
+
- The model was fine yesterday but is hallucinating today
|
|
26
|
+
- You suspect hidden repair/retry loops silently mutating responses
|
|
27
|
+
|
|
28
|
+
**Do not use for:**
|
|
29
|
+
- General code debugging — use `agent-introspection-debugging`
|
|
30
|
+
- Code review — use language-specific reviewer agents
|
|
31
|
+
- Security scanning — use `security-review` or `security-review/scan`
|
|
32
|
+
- Agent performance benchmarking — use `agent-eval`
|
|
33
|
+
- Writing new features — use the appropriate workflow skill
|
|
34
|
+
|
|
35
|
+
## The 12-Layer Stack
|
|
36
|
+
|
|
37
|
+
Every agent system has these layers. Any of them can corrupt the answer:
|
|
38
|
+
|
|
39
|
+
| # | Layer | What Goes Wrong |
|
|
40
|
+
|---|-------|----------------|
|
|
41
|
+
| 1 | System prompt | Conflicting instructions, instruction bloat |
|
|
42
|
+
| 2 | Session history | Stale context injection from previous turns |
|
|
43
|
+
| 3 | Long-term memory | Pollution across sessions, old topics in new conversations |
|
|
44
|
+
| 4 | Distillation | Compressed artifacts re-entering as pseudo-facts |
|
|
45
|
+
| 5 | Active recall | Redundant re-summary layers wasting context |
|
|
46
|
+
| 6 | Tool selection | Wrong tool routing, model skips required tools |
|
|
47
|
+
| 7 | Tool execution | Hallucinated execution — claims to call but doesn't |
|
|
48
|
+
| 8 | Tool interpretation | Misread or ignored tool output |
|
|
49
|
+
| 9 | Answer shaping | Format corruption in final response |
|
|
50
|
+
| 10 | Platform rendering | Transport-layer mutation (UI, API, CLI mutates valid answers) |
|
|
51
|
+
| 11 | Hidden repair loops | Silent fallback/retry agents running second LLM pass |
|
|
52
|
+
| 12 | Persistence | Expired state or cached artifacts reused as live evidence |
|
|
53
|
+
|
|
54
|
+
## Common Failure Patterns
|
|
55
|
+
|
|
56
|
+
### 1. Wrapper Regression
|
|
57
|
+
|
|
58
|
+
The base model produces correct answers, but the wrapper layers make it worse.
|
|
59
|
+
|
|
60
|
+
**Symptoms:**
|
|
61
|
+
- Model works fine in playground or direct API call, breaks in your agent
|
|
62
|
+
- Added a new prompt layer, existing behavior degraded
|
|
63
|
+
- Agent sounds confident but is confidently wrong
|
|
64
|
+
- "It was working before the last update"
|
|
65
|
+
|
|
66
|
+
### 2. Memory Contamination
|
|
67
|
+
|
|
68
|
+
Old topics leak into new conversations through history, memory retrieval, or distillation.
|
|
69
|
+
|
|
70
|
+
**Symptoms:**
|
|
71
|
+
- Agent brings up unrelated past topics
|
|
72
|
+
- User corrections don't stick (old memory overwrites new)
|
|
73
|
+
- Same-session artifacts re-enter as pseudo-facts
|
|
74
|
+
- Memory grows without bound, degrading response quality over time
|
|
75
|
+
|
|
76
|
+
### 3. Tool Discipline Failure
|
|
77
|
+
|
|
78
|
+
Tools are declared in the prompt but not enforced in code. The model skips them or hallucinates execution.
|
|
79
|
+
|
|
80
|
+
**Symptoms:**
|
|
81
|
+
- "Must use tool X" in prompt, but model answers without calling it
|
|
82
|
+
- Tool results look correct but were never actually executed
|
|
83
|
+
- Different tools fight over the same responsibility
|
|
84
|
+
- Model uses tool when it shouldn't, or skips it when it must
|
|
85
|
+
|
|
86
|
+
### 4. Rendering/Transport Corruption
|
|
87
|
+
|
|
88
|
+
The agent's internal answer is correct, but the platform layer mutates it during delivery.
|
|
89
|
+
|
|
90
|
+
**Symptoms:**
|
|
91
|
+
- Logs show correct answer, user sees broken output
|
|
92
|
+
- Markdown rendering, JSON parsing, or streaming fragments corrupt valid responses
|
|
93
|
+
- Hidden fallback agent quietly replaces the answer before delivery
|
|
94
|
+
- Output differs between terminal and UI
|
|
95
|
+
|
|
96
|
+
### 5. Hidden Agent Layers
|
|
97
|
+
|
|
98
|
+
Silent repair, retry, summarization, or recall agents run without explicit contracts.
|
|
99
|
+
|
|
100
|
+
**Symptoms:**
|
|
101
|
+
- Output changes between internal generation and user delivery
|
|
102
|
+
- "Auto-fix" loops run a second LLM pass the user doesn't know about
|
|
103
|
+
- Multiple agents modify the same output without coordination
|
|
104
|
+
- Answers get "smoothed" or "corrected" by invisible layers
|
|
105
|
+
|
|
106
|
+
## Audit Workflow
|
|
107
|
+
|
|
108
|
+
### Phase 1: Scope
|
|
109
|
+
|
|
110
|
+
Define what you're auditing:
|
|
111
|
+
|
|
112
|
+
- **Target system** — what agent application?
|
|
113
|
+
- **Entrypoints** — how do users interact with it?
|
|
114
|
+
- **Model stack** — which LLM(s) and providers?
|
|
115
|
+
- **Symptoms** — what does the user report?
|
|
116
|
+
- **Time window** — when did it start?
|
|
117
|
+
- **Layers to audit** — which of the 12 layers apply?
|
|
118
|
+
|
|
119
|
+
### Phase 2: Evidence Collection
|
|
120
|
+
|
|
121
|
+
Gather evidence from the codebase:
|
|
122
|
+
|
|
123
|
+
- **Source code** — agent loop, tool router, memory admission, prompt assembly
|
|
124
|
+
- **Logs** — historical session traces, tool call records
|
|
125
|
+
- **Config** — prompt templates, tool schemas, provider settings
|
|
126
|
+
- **Memory files** — SOPs, knowledge bases, session archives
|
|
127
|
+
|
|
128
|
+
Use `rg` to search for anti-patterns:
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
# Tool requirements expressed only in prompt text (not code)
|
|
132
|
+
rg "must.*tool|必须.*工具|required.*call" --type md
|
|
133
|
+
|
|
134
|
+
# Tool execution without validation
|
|
135
|
+
rg "tool_call|toolCall|tool_use" --type py --type ts
|
|
136
|
+
|
|
137
|
+
# Hidden LLM calls outside main agent loop
|
|
138
|
+
rg "completion|chat\.create|messages\.create|llm\.invoke"
|
|
139
|
+
|
|
140
|
+
# Memory admission without user-correction priority
|
|
141
|
+
rg "memory.*admit|long.*term.*update|persist.*memory" --type py --type ts
|
|
142
|
+
|
|
143
|
+
# Fallback loops that run additional LLM calls
|
|
144
|
+
rg "fallback|retry.*llm|repair.*prompt|re-?prompt" --type py --type ts
|
|
145
|
+
|
|
146
|
+
# Silent output mutation
|
|
147
|
+
rg "mutate|rewrite.*response|transform.*output|shap" --type py --type ts
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
### Phase 3: Failure Mapping
|
|
151
|
+
|
|
152
|
+
For each finding, document:
|
|
153
|
+
|
|
154
|
+
- **Symptom** — what the user sees
|
|
155
|
+
- **Mechanism** — how the wrapper causes it
|
|
156
|
+
- **Source layer** — which of the 12 layers
|
|
157
|
+
- **Root cause** — the deepest cause
|
|
158
|
+
- **Evidence** — file:line or log:row reference
|
|
159
|
+
- **Confidence** — 0.0 to 1.0
|
|
160
|
+
|
|
161
|
+
### Phase 4: Fix Strategy
|
|
162
|
+
|
|
163
|
+
Default fix order (code-first, not prompt-first):
|
|
164
|
+
|
|
165
|
+
1. **Code-gate tool requirements** — enforce in code, not just prompt text
|
|
166
|
+
2. **Remove or narrow hidden repair agents** — make fallback explicit with contracts
|
|
167
|
+
3. **Reduce context duplication** — same info through prompt + history + memory + distillation
|
|
168
|
+
4. **Tighten memory admission** — user corrections > agent assertions
|
|
169
|
+
5. **Tighten distillation triggers** — don't compress what shouldn't be compressed
|
|
170
|
+
6. **Reduce rendering mutation** — pass-through, don't transform
|
|
171
|
+
7. **Convert to typed JSON envelopes** — structured internal flow, not freeform prose
|
|
172
|
+
|
|
173
|
+
## Severity Model
|
|
174
|
+
|
|
175
|
+
| Level | Meaning | Action |
|
|
176
|
+
|-------|---------|--------|
|
|
177
|
+
| `critical` | Agent can confidently produce wrong operational behavior | Fix before next release |
|
|
178
|
+
| `high` | Agent frequently degrades correctness or stability | Fix this sprint |
|
|
179
|
+
| `medium` | Correctness usually survives but output is fragile or wasteful | Plan for next cycle |
|
|
180
|
+
| `low` | Mostly cosmetic or maintainability issues | Backlog |
|
|
181
|
+
|
|
182
|
+
## Output Format
|
|
183
|
+
|
|
184
|
+
Present findings to the user in this order:
|
|
185
|
+
|
|
186
|
+
1. **Severity-ranked findings** (most critical first)
|
|
187
|
+
2. **Architecture diagnosis** (which layer corrupted what, and why)
|
|
188
|
+
3. **Ordered fix plan** (code-first, not prompt-first)
|
|
189
|
+
|
|
190
|
+
Do not lead with compliments or summaries. If the system is broken, say so directly.
|
|
191
|
+
|
|
192
|
+
## Quick Diagnostic Questions
|
|
193
|
+
|
|
194
|
+
When auditing an agent system, answer these:
|
|
195
|
+
|
|
196
|
+
| # | Question | If Yes → |
|
|
197
|
+
|---|----------|----------|
|
|
198
|
+
| 1 | Can the model skip a required tool and still answer? | Tool not code-gated |
|
|
199
|
+
| 2 | Does old conversation content appear in new turns? | Memory contamination |
|
|
200
|
+
| 3 | Is the same info in system prompt AND memory AND history? | Context duplication |
|
|
201
|
+
| 4 | Does the platform run a second LLM pass before delivery? | Hidden repair loop |
|
|
202
|
+
| 5 | Does the output differ between internal generation and user delivery? | Rendering corruption |
|
|
203
|
+
| 6 | Are "must use tool X" rules only in prompt text? | Tool discipline failure |
|
|
204
|
+
| 7 | Can the agent's own monologue become persistent memory? | Memory poisoning |
|
|
205
|
+
|
|
206
|
+
## Anti-Patterns to Avoid
|
|
207
|
+
|
|
208
|
+
- Avoid blaming the model before falsifying wrapper-layer regressions.
|
|
209
|
+
- Avoid blaming memory without showing the contamination path.
|
|
210
|
+
- Do not let a clean current state erase a dirty historical incident.
|
|
211
|
+
- Do not treat markdown prose as a trustworthy internal protocol.
|
|
212
|
+
- Do not accept "must use tool" in prompt text when code never enforces it.
|
|
213
|
+
- Keep findings direct, evidence-backed, and severity-ranked.
|
|
214
|
+
|
|
215
|
+
## Report Schema
|
|
216
|
+
|
|
217
|
+
Audits should produce structured reports following this shape:
|
|
218
|
+
|
|
219
|
+
```json
|
|
220
|
+
{
|
|
221
|
+
"schema_version": "ecc.agent-architecture-audit.report.v1",
|
|
222
|
+
"executive_verdict": {
|
|
223
|
+
"overall_health": "high_risk",
|
|
224
|
+
"primary_failure_mode": "string",
|
|
225
|
+
"most_urgent_fix": "string"
|
|
226
|
+
},
|
|
227
|
+
"scope": {
|
|
228
|
+
"target_name": "string",
|
|
229
|
+
"model_stack": ["string"],
|
|
230
|
+
"layers_to_audit": ["string"]
|
|
231
|
+
},
|
|
232
|
+
"findings": [
|
|
233
|
+
{
|
|
234
|
+
"severity": "critical|high|medium|low",
|
|
235
|
+
"title": "string",
|
|
236
|
+
"mechanism": "string",
|
|
237
|
+
"source_layer": "string",
|
|
238
|
+
"root_cause": "string",
|
|
239
|
+
"evidence_refs": ["file:line"],
|
|
240
|
+
"confidence": 0.0,
|
|
241
|
+
"recommended_fix": "string"
|
|
242
|
+
}
|
|
243
|
+
],
|
|
244
|
+
"ordered_fix_plan": [
|
|
245
|
+
{ "order": 1, "goal": "string", "why_now": "string", "expected_effect": "string" }
|
|
246
|
+
]
|
|
247
|
+
}
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
## Related Skills
|
|
251
|
+
|
|
252
|
+
- `agent-introspection-debugging` — Debug agent runtime failures (loops, timeouts, state errors)
|
|
253
|
+
- `agent-eval` — Benchmark agent performance head-to-head
|
|
254
|
+
- `security-review` — Security audit for code and configuration
|
|
255
|
+
- `autonomous-agent-harness` — Set up autonomous agent operations
|
|
256
|
+
- `agent-harness-construction` — Build agent harnesses from scratch
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: agent-harness-construction
|
|
3
|
+
description: Design and optimize AI agent action spaces, tool definitions, and observation formatting for higher completion rates.
|
|
4
|
+
origin: ECC
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Agent Harness Construction
|
|
8
|
+
|
|
9
|
+
Use this skill when you are improving how an agent plans, calls tools, recovers from errors, and converges on completion.
|
|
10
|
+
|
|
11
|
+
## Core Model
|
|
12
|
+
|
|
13
|
+
Agent output quality is constrained by:
|
|
14
|
+
1. Action space quality
|
|
15
|
+
2. Observation quality
|
|
16
|
+
3. Recovery quality
|
|
17
|
+
4. Context budget quality
|
|
18
|
+
|
|
19
|
+
## Action Space Design
|
|
20
|
+
|
|
21
|
+
1. Use stable, explicit tool names.
|
|
22
|
+
2. Keep inputs schema-first and narrow.
|
|
23
|
+
3. Return deterministic output shapes.
|
|
24
|
+
4. Avoid catch-all tools unless isolation is impossible.
|
|
25
|
+
|
|
26
|
+
## Granularity Rules
|
|
27
|
+
|
|
28
|
+
- Use micro-tools for high-risk operations (deploy, migration, permissions).
|
|
29
|
+
- Use medium tools for common edit/read/search loops.
|
|
30
|
+
- Use macro-tools only when round-trip overhead is the dominant cost.
|
|
31
|
+
|
|
32
|
+
## Observation Design
|
|
33
|
+
|
|
34
|
+
Every tool response should include:
|
|
35
|
+
- `status`: success|warning|error
|
|
36
|
+
- `summary`: one-line result
|
|
37
|
+
- `next_actions`: actionable follow-ups
|
|
38
|
+
- `artifacts`: file paths / IDs
|
|
39
|
+
|
|
40
|
+
## Error Recovery Contract
|
|
41
|
+
|
|
42
|
+
For every error path, include:
|
|
43
|
+
- root cause hint
|
|
44
|
+
- safe retry instruction
|
|
45
|
+
- explicit stop condition
|
|
46
|
+
|
|
47
|
+
## Context Budgeting
|
|
48
|
+
|
|
49
|
+
1. Keep system prompt minimal and invariant.
|
|
50
|
+
2. Move large guidance into skills loaded on demand.
|
|
51
|
+
3. Prefer references to files over inlining long documents.
|
|
52
|
+
4. Compact at phase boundaries, not arbitrary token thresholds.
|
|
53
|
+
|
|
54
|
+
## Architecture Pattern Guidance
|
|
55
|
+
|
|
56
|
+
- ReAct: best for exploratory tasks with uncertain path.
|
|
57
|
+
- Function-calling: best for structured deterministic flows.
|
|
58
|
+
- Hybrid (recommended): ReAct planning + typed tool execution.
|
|
59
|
+
|
|
60
|
+
## Benchmarking
|
|
61
|
+
|
|
62
|
+
Track:
|
|
63
|
+
- completion rate
|
|
64
|
+
- retries per task
|
|
65
|
+
- pass@1 and pass@3
|
|
66
|
+
- cost per successful task
|
|
67
|
+
|
|
68
|
+
## Anti-Patterns
|
|
69
|
+
|
|
70
|
+
- Too many tools with overlapping semantics.
|
|
71
|
+
- Opaque tool output with no recovery hints.
|
|
72
|
+
- Error-only output without next steps.
|
|
73
|
+
- Context overloading with irrelevant references.
|