opencode-skills-collection 3.0.45 → 3.0.47
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled-skills/.antigravity-install-manifest.json +10 -1
- package/bundled-skills/2slides-ppt-generator/SKILL.md +1 -1
- package/bundled-skills/2slides-ppt-generator/scripts/create_pdf_slides.py +2 -1
- package/bundled-skills/2slides-ppt-generator/scripts/generate_narration.py +2 -1
- package/bundled-skills/2slides-ppt-generator/scripts/generate_slides.py +13 -7
- package/bundled-skills/android-dev/references/hybrid.md +7 -4
- package/bundled-skills/android-dev/references/react-native.md +5 -2
- package/bundled-skills/atlas-contract/SKILL.md +4 -4
- package/bundled-skills/atlas-ledger/SKILL.md +10 -7
- package/bundled-skills/bun-development/SKILL.md +1 -1
- package/bundled-skills/cloud-penetration-testing/SKILL.md +1 -1
- package/bundled-skills/codebase-to-wordpress-converter/SKILL.md +1 -0
- package/bundled-skills/docs/integrations/jetski-cortex.md +3 -3
- package/bundled-skills/docs/integrations/jetski-gemini-loader/README.md +1 -1
- package/bundled-skills/docs/maintainers/repo-growth-seo.md +3 -3
- package/bundled-skills/docs/maintainers/skills-update-guide.md +1 -1
- package/bundled-skills/docs/users/bundles.md +1 -1
- package/bundled-skills/docs/users/claude-code-skills.md +1 -1
- package/bundled-skills/docs/users/gemini-cli-skills.md +1 -1
- package/bundled-skills/docs/users/getting-started.md +1 -1
- package/bundled-skills/docs/users/kiro-integration.md +1 -1
- package/bundled-skills/docs/users/usage.md +4 -4
- package/bundled-skills/docs/users/visual-guide.md +4 -4
- package/bundled-skills/dos-verify-done-claims/SKILL.md +173 -0
- package/bundled-skills/ecl-harness-engineer/LICENSE +21 -0
- package/bundled-skills/ecl-harness-engineer/SKILL.md +714 -0
- package/bundled-skills/ecl-harness-engineer/agents/analyzer.md +119 -0
- package/bundled-skills/ecl-harness-engineer/agents/auditor.md +212 -0
- package/bundled-skills/ecl-harness-engineer/agents/creator-config.md +343 -0
- package/bundled-skills/ecl-harness-engineer/agents/creator-docs.md +201 -0
- package/bundled-skills/ecl-harness-engineer/agents/creator-linters.md +123 -0
- package/bundled-skills/ecl-harness-engineer/references/adapters/adapter-schema.md +204 -0
- package/bundled-skills/ecl-harness-engineer/references/adapters/generic.md +156 -0
- package/bundled-skills/ecl-harness-engineer/references/adapters/go.md +212 -0
- package/bundled-skills/ecl-harness-engineer/references/adapters/java.md +205 -0
- package/bundled-skills/ecl-harness-engineer/references/adapters/python.md +225 -0
- package/bundled-skills/ecl-harness-engineer/references/adapters/rust.md +220 -0
- package/bundled-skills/ecl-harness-engineer/references/adapters/typescript.md +245 -0
- package/bundled-skills/ecl-harness-engineer/references/architecture-diagrams.md +420 -0
- package/bundled-skills/ecl-harness-engineer/references/audit-templates.md +649 -0
- package/bundled-skills/ecl-harness-engineer/references/capability-registry.md +485 -0
- package/bundled-skills/ecl-harness-engineer/references/darwin-eval-prompts.md +373 -0
- package/bundled-skills/ecl-harness-engineer/references/documentation-templates.md +741 -0
- package/bundled-skills/ecl-harness-engineer/references/durability-patterns.md +423 -0
- package/bundled-skills/ecl-harness-engineer/references/ecl-harness.md +1431 -0
- package/bundled-skills/ecl-harness-engineer/references/environment-config-guide.md +534 -0
- package/bundled-skills/ecl-harness-engineer/references/environment-detection-guide.md +751 -0
- package/bundled-skills/ecl-harness-engineer/references/eval-templates.md +377 -0
- package/bundled-skills/ecl-harness-engineer/references/gc-templates.md +798 -0
- package/bundled-skills/ecl-harness-engineer/references/greenfield-templates.md +1385 -0
- package/bundled-skills/ecl-harness-engineer/references/linter-templates.md +448 -0
- package/bundled-skills/ecl-harness-engineer/references/observability-templates.md +315 -0
- package/bundled-skills/environment-setup-guide/SKILL.md +2 -2
- package/bundled-skills/evolution/SKILL.md +1 -1
- package/bundled-skills/gitops-workflow/SKILL.md +1 -1
- package/bundled-skills/linkerd-patterns/SKILL.md +1 -1
- package/bundled-skills/loki-mode/examples/todo-app-generated/frontend/package-lock.json +504 -1317
- package/bundled-skills/loki-mode/examples/todo-app-generated/frontend/package.json +2 -2
- package/bundled-skills/lovable-cleanup/SKILL.md +416 -0
- package/bundled-skills/monopoly/SKILL.md +397 -0
- package/bundled-skills/monopoly/patterns/SKILL.md +331 -0
- package/bundled-skills/monopoly/scale-benchmarks/SKILL.md +174 -0
- package/bundled-skills/monopoly/security-checklist/SKILL.md +69 -0
- package/bundled-skills/monopoly/tech-matrix/SKILL.md +268 -0
- package/bundled-skills/pagespeed-enhancer/SKILL.md +579 -0
- package/bundled-skills/polis-protocol/SKILL.md +6 -3
- package/bundled-skills/unship/SKILL.md +11 -5
- package/bundled-skills/uv-package-manager/resources/implementation-playbook.md +1 -1
- package/bundled-skills/varlock/SKILL.md +2 -2
- package/package.json +1 -1
- package/skills_index.json +204 -4
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
# Code Architecture Analysis Agent
|
|
2
|
+
|
|
3
|
+
You are analyzing a codebase to understand its architecture for building agent harness infrastructure.
|
|
4
|
+
|
|
5
|
+
## Your Task
|
|
6
|
+
|
|
7
|
+
Produce a complete architectural analysis that can be used by other agents to create documentation, linters, and configuration.
|
|
8
|
+
|
|
9
|
+
## Step-by-Step
|
|
10
|
+
|
|
11
|
+
### 1. Identify Tech Stack
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
ls go.mod package.json requirements.txt pyproject.toml Cargo.toml 2>/dev/null
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Record: language, version, key dependencies.
|
|
18
|
+
|
|
19
|
+
### 2. Map Directory Structure
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
find . -type f \( -name "*.go" -o -name "*.ts" -o -name "*.js" -o -name "*.py" -o -name "*.rs" \) \
|
|
23
|
+
! -path './.git/*' ! -path './node_modules/*' ! -path './vendor/*' | head -100
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Identify the organizational pattern (cmd/ + internal/, src/ + lib/, etc.)
|
|
27
|
+
|
|
28
|
+
### 3. Build Layer Hierarchy from Imports
|
|
29
|
+
|
|
30
|
+
This is the most critical step. Analyze actual import relationships:
|
|
31
|
+
|
|
32
|
+
**Go**: `grep -r '"module-path/' --include="*.go"` or `go list -json ./...`
|
|
33
|
+
**TypeScript**: `grep -r "from ['\"]\.\.?/" --include="*.ts" --include="*.tsx"`
|
|
34
|
+
**Python**: `grep -r "^from \." --include="*.py"`
|
|
35
|
+
|
|
36
|
+
Assign layers bottom-up:
|
|
37
|
+
- Layer 0: Packages with ZERO internal imports
|
|
38
|
+
- Layer N: Packages that only import from layers < N
|
|
39
|
+
|
|
40
|
+
Record every package and its layer assignment.
|
|
41
|
+
|
|
42
|
+
### 4. Detect Circular Dependencies
|
|
43
|
+
|
|
44
|
+
If Package A imports Package B AND Package B imports Package A → P0 issue.
|
|
45
|
+
|
|
46
|
+
Record:
|
|
47
|
+
- Files involved (with line numbers)
|
|
48
|
+
- Type: direct vs transitive
|
|
49
|
+
- Suggested fix
|
|
50
|
+
|
|
51
|
+
### 5. Extract Key Interfaces
|
|
52
|
+
|
|
53
|
+
Search for interface/abstract definitions:
|
|
54
|
+
- Go: `grep -r "type.*interface" --include="*.go"`
|
|
55
|
+
- TypeScript: `grep -r "interface\|abstract class" --include="*.ts"`
|
|
56
|
+
- Python: `grep -r "@abstractmethod" --include="*.py"`
|
|
57
|
+
|
|
58
|
+
For each key interface, record: name, location (file:line), methods, implementations, usage sites.
|
|
59
|
+
|
|
60
|
+
### 6. Trace Critical Code Paths
|
|
61
|
+
|
|
62
|
+
Pick 3-5 representative paths (happy path, error path, complex flow, background job).
|
|
63
|
+
|
|
64
|
+
For each, trace from entry point through all layers:
|
|
65
|
+
```
|
|
66
|
+
[file:line] function_name()
|
|
67
|
+
↓ calls
|
|
68
|
+
[file:line] another_function()
|
|
69
|
+
↓ returns
|
|
70
|
+
...
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
### 7. Catalog Error Handling Patterns
|
|
74
|
+
|
|
75
|
+
Identify:
|
|
76
|
+
- Typed errors vs strings?
|
|
77
|
+
- Error wrapping convention?
|
|
78
|
+
- Error code registry?
|
|
79
|
+
- Structured logging?
|
|
80
|
+
- Retry logic?
|
|
81
|
+
|
|
82
|
+
## Output Format
|
|
83
|
+
|
|
84
|
+
Save results to `harness/.analysis/architecture.json`:
|
|
85
|
+
|
|
86
|
+
```json
|
|
87
|
+
{
|
|
88
|
+
"tech_stack": {
|
|
89
|
+
"language": "Go",
|
|
90
|
+
"version": "1.22",
|
|
91
|
+
"module_path": "github.com/org/project",
|
|
92
|
+
"key_dependencies": ["chi", "pgx", "zap"]
|
|
93
|
+
},
|
|
94
|
+
"layers": [
|
|
95
|
+
{"level": 0, "packages": ["internal/types", "internal/errors"], "description": "Core types, zero internal deps"},
|
|
96
|
+
{"level": 1, "packages": ["internal/utils", "internal/logging"], "description": "Utilities, only imports L0"},
|
|
97
|
+
{"level": 2, "packages": ["internal/core", "internal/auth"], "description": "Business logic"}
|
|
98
|
+
],
|
|
99
|
+
"circular_dependencies": [
|
|
100
|
+
{"pkg_a": "internal/auth", "pkg_b": "internal/core", "files": ["auth/middleware.go:15", "core/service.go:23"], "suggested_fix": "Extract shared interface"}
|
|
101
|
+
],
|
|
102
|
+
"key_interfaces": [
|
|
103
|
+
{"name": "UserService", "location": "internal/core/user.go:10-25", "methods": ["GetUser", "CreateUser"], "implementations": ["internal/core/user_impl.go"]}
|
|
104
|
+
],
|
|
105
|
+
"code_paths": [
|
|
106
|
+
{"name": "Create User", "trigger": "POST /api/users", "flow": ["cmd/api.go:45", "core/user.go:30", "storage/user.go:15"]}
|
|
107
|
+
],
|
|
108
|
+
"error_patterns": {
|
|
109
|
+
"style": "typed_errors",
|
|
110
|
+
"wrapping": true,
|
|
111
|
+
"structured_logging": true,
|
|
112
|
+
"error_registry": "internal/errors/codes.go"
|
|
113
|
+
},
|
|
114
|
+
"total_files": 45,
|
|
115
|
+
"total_lines": 3500
|
|
116
|
+
}
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
Also write a human-readable summary to `harness/.analysis/architecture-summary.md`.
|
|
@@ -0,0 +1,212 @@
|
|
|
1
|
+
# Harness State Audit Agent
|
|
2
|
+
|
|
3
|
+
You are auditing the existing harness infrastructure of a codebase to identify gaps and issues.
|
|
4
|
+
|
|
5
|
+
## Your Task
|
|
6
|
+
|
|
7
|
+
Produce a comprehensive audit report showing what exists, what's missing, and what's broken.
|
|
8
|
+
|
|
9
|
+
## Profile Detection
|
|
10
|
+
|
|
11
|
+
Audit the project as `core` unless the repository or user request explicitly enables advanced
|
|
12
|
+
agent-platform capabilities such as agent evals, execution traces, long-term memory, checkpoints,
|
|
13
|
+
or metrics.
|
|
14
|
+
|
|
15
|
+
- **Core profile**: score documentation, linters, environment/config, integration, and ECL change
|
|
16
|
+
system, including lightweight auto-evolve threshold checking. Do not penalize missing `harness/eval`, `harness/trace`, `harness/memory`,
|
|
17
|
+
`harness/checkpoints`, or `harness/metrics`.
|
|
18
|
+
- **Advanced profile**: run the core audit plus the advanced eval and quality automation checks.
|
|
19
|
+
|
|
20
|
+
## Audit Dimensions
|
|
21
|
+
|
|
22
|
+
### 1. Documentation (Weight: 25%)
|
|
23
|
+
|
|
24
|
+
| Check | How | Pass Criteria |
|
|
25
|
+
|-------|-----|---------------|
|
|
26
|
+
| AGENTS.md exists | `test -f AGENTS.md` | File exists |
|
|
27
|
+
| AGENTS.md size | `wc -l AGENTS.md` | 80-120 lines |
|
|
28
|
+
| AGENTS.md has numbered sections | Count `##` headers | ≥ 5 sections |
|
|
29
|
+
| ARCHITECTURE.md exists | `test -f docs/ARCHITECTURE.md` | File exists |
|
|
30
|
+
| ARCHITECTURE.md has Mermaid diagrams | `grep 'mermaid' docs/ARCHITECTURE.md` | At least 1 |
|
|
31
|
+
| Layer claims are accurate | Cross-reference imports | No false claims |
|
|
32
|
+
| DEVELOPMENT.md commands work | Spot-check 2-3 commands | Commands succeed |
|
|
33
|
+
| Design docs exist (not just index) | `find docs/design-docs -name "*.md" ! -name "index.md"` | ≥ 2 files |
|
|
34
|
+
| All doc links are valid | Check `[text](path)` references | No broken links |
|
|
35
|
+
| ECL doc exists | `test -f docs/ECL.md` | File exists |
|
|
36
|
+
| ECL doc defines lifecycle | Read docs/ECL.md | active/parking/archive and update protocol documented |
|
|
37
|
+
| STATUS handoff exists | `test -f docs/STATUS.md` | File exists when ECL is enabled |
|
|
38
|
+
| STATUS priority is correct | Read docs/STATUS.md and AGENTS.md | Active change overrides STATUS; STATUS is used only when no active exists |
|
|
39
|
+
|
|
40
|
+
### 2. Linters (Weight: 20%)
|
|
41
|
+
|
|
42
|
+
| Check | How | Pass Criteria |
|
|
43
|
+
|-------|-----|---------------|
|
|
44
|
+
| lint-deps script exists | `test -f scripts/lint-deps*` | File exists |
|
|
45
|
+
| lint-quality script exists | `test -f scripts/lint-quality*` | File exists |
|
|
46
|
+
| Layer map covers all packages | Compare map vs `go list ./...` | 100% coverage |
|
|
47
|
+
| Can detect real violations | Create test case | Violation caught |
|
|
48
|
+
| Error messages are agent-actionable | Read 5 error messages | WHAT + WHY + HOW |
|
|
49
|
+
| `make lint-arch` passes | Run it | Exit code 0 |
|
|
50
|
+
|
|
51
|
+
### 3. Eval System (Advanced profile only; Weight: 20% when enabled)
|
|
52
|
+
|
|
53
|
+
| Check | How | Pass Criteria |
|
|
54
|
+
|-------|-----|---------------|
|
|
55
|
+
| Eval directory exists | `test -d harness/eval` | Directory exists |
|
|
56
|
+
| Eval datasets present | `find harness/eval/datasets -name "*.json"` | ≥ 5 tasks |
|
|
57
|
+
| Categories covered | Count unique categories | ≥ 3 |
|
|
58
|
+
| Tasks reference real files | Spot-check file paths | Valid references |
|
|
59
|
+
| Task freshness | Check git dates | Updated within 90 days |
|
|
60
|
+
|
|
61
|
+
### 4. Environment & Config (Weight: 15%)
|
|
62
|
+
|
|
63
|
+
| Check | How | Pass Criteria |
|
|
64
|
+
|-------|-----|---------------|
|
|
65
|
+
| environment.json exists | `test -f harness/config/environment.json` | File exists (if project has external deps) |
|
|
66
|
+
| Setup scripts exist | `test -f harness/scripts/setup-env.sh` | File exists |
|
|
67
|
+
| Scripts are executable | `test -x harness/scripts/*.sh` | Executable |
|
|
68
|
+
| No hardcoded secrets | `grep -r "password\|secret\|key=" harness/config/` | Uses ${VAR} references |
|
|
69
|
+
|
|
70
|
+
### 5. Integration (Weight: 10%)
|
|
71
|
+
|
|
72
|
+
| Check | How | Pass Criteria |
|
|
73
|
+
|-------|-----|---------------|
|
|
74
|
+
| Makefile has lint-arch target | `grep 'lint-arch' Makefile` | Target exists |
|
|
75
|
+
| Build passes | `make build` or equivalent | Exit code 0 |
|
|
76
|
+
| CI config exists | `test -f .github/workflows/ci.yml` | File exists |
|
|
77
|
+
|
|
78
|
+
### 6. Quality Automation (Advanced profile only; Weight: 10% when enabled)
|
|
79
|
+
|
|
80
|
+
| Check | How | Pass Criteria |
|
|
81
|
+
|-------|-----|---------------|
|
|
82
|
+
| Observability structure | `test -d harness/trace` | Directory exists |
|
|
83
|
+
| Memory structure | `test -d harness/memory` | Directory exists |
|
|
84
|
+
| Checkpointing support | `test -d harness/checkpoints` | Directory exists |
|
|
85
|
+
|
|
86
|
+
### 7. ECL Change System (Weight: report separately)
|
|
87
|
+
|
|
88
|
+
| Check | How | Pass Criteria |
|
|
89
|
+
|-------|-----|---------------|
|
|
90
|
+
| changes directories exist | `test -d harness/changes/active && test -d harness/changes/parking && test -d harness/changes/archive` | Directories exist |
|
|
91
|
+
| change templates exist | `test -f harness/templates/change/summary.md` etc. | New harnesses have summary/spec/plan/tasks/reviews templates; old archives may remain 4-file |
|
|
92
|
+
| harness-change script exists | `test -f scripts/harness-change.*` | One selected command-surface implementation exists |
|
|
93
|
+
| lint-ecl exists | `test -f scripts/lint-ecl.*` | One selected command-surface implementation exists |
|
|
94
|
+
| lint-encoding exists | `test -f scripts/lint-encoding.*` | One selected command-surface implementation exists |
|
|
95
|
+
| INDEX.json is generated | Run generated `harness-change reindex` command or dry-run equivalent | Index matches parking/archive |
|
|
96
|
+
| active is single | Inspect changes dir | No multiple active task directories |
|
|
97
|
+
| archive loading is selective | Read AGENTS.md/docs/ECL.md | History loads through STATUS/INDEX; no default full archive load |
|
|
98
|
+
|
|
99
|
+
### 8. Auto-Evolve (Core profile; Weight: report separately)
|
|
100
|
+
|
|
101
|
+
| Check | How | Pass Criteria |
|
|
102
|
+
|-------|-----|---------------|
|
|
103
|
+
| evolution state exists | `test -f harness/evolution/state.json` | File exists with enabled, threshold, window, last_evolved_archive_count |
|
|
104
|
+
| harness-evolve script exists | `test -f scripts/harness-evolve.*` | One selected command-surface implementation exists |
|
|
105
|
+
| close/reindex trigger check | Read `scripts/harness-change.*` | `close` and `reindex` run `harness-evolve check` or equivalent |
|
|
106
|
+
| pending is bounded | Read generated docs/scripts | pending lists candidate archive summaries, not full archive contents |
|
|
107
|
+
| active work has priority | Read AGENTS.md/docs/ECL.md | pending is deferred when active change exists |
|
|
108
|
+
| no advanced dirs by default | Inspect harness tree | no eval/trace/memory/checkpoints/metrics unless explicitly requested |
|
|
109
|
+
| ratchet rule documented | Read docs/ECL.md | keep only if score improves and verification passes; otherwise revert |
|
|
110
|
+
| independent scoring documented | Read docs/ECL.md and proposals | auto-apply requires an auditor/subagent independent review |
|
|
111
|
+
| proposal-first flow | Inspect `harness/evolution/proposals/` | accepted/rejected candidates are separated before file edits |
|
|
112
|
+
| results log decisions | Read `harness/evolution/results.tsv` | status is one of keep/revert/rejected/noop and eval_mode is present |
|
|
113
|
+
|
|
114
|
+
## Auto-Evolve Independent Review
|
|
115
|
+
|
|
116
|
+
When asked to score an auto-evolve proposal, act as an independent evaluator. Do not generate or
|
|
117
|
+
edit the delta you are scoring. Return a concise decision object and a short explanation.
|
|
118
|
+
|
|
119
|
+
Score out of 100:
|
|
120
|
+
|
|
121
|
+
| Dimension | Weight | Pass Criteria |
|
|
122
|
+
|-----------|-------:|---------------|
|
|
123
|
+
| Evidence grounding | 30 | Accepted candidates cite specific archived summaries, reviews, or validation notes |
|
|
124
|
+
| Project relevance | 25 | Accepted candidates map to current project modules, files, commands, failures, or user corrections |
|
|
125
|
+
| Mechanical enforceability | 15 | Important rules become lint/test/CI checks or explicit acceptance gates |
|
|
126
|
+
| Regression safety | 20 | Proposed delta does not weaken harness checks or business gates |
|
|
127
|
+
| Context cost | 10 | AGENTS.md stays concise and archive loading remains bounded |
|
|
128
|
+
|
|
129
|
+
Hard rejection conditions:
|
|
130
|
+
|
|
131
|
+
- No archived change evidence for an accepted candidate.
|
|
132
|
+
- Candidate is generic best practice, article advice, or model inference without project evidence.
|
|
133
|
+
- Candidate cannot name affected project files, modules, commands, failures, or user corrections.
|
|
134
|
+
- Candidate would default-create `harness/eval`, `harness/trace`, `harness/state`,
|
|
135
|
+
`harness/checkpoints`, `harness/memory`, or `harness/metrics`.
|
|
136
|
+
- Candidate would put rejected material into AGENTS.md, ECL, STATUS, lint, or CI.
|
|
137
|
+
|
|
138
|
+
Decision rules:
|
|
139
|
+
|
|
140
|
+
- `keep`: score >= 80, hard gates pass, and validation plan is adequate.
|
|
141
|
+
- `rejected`: hard gate fails or score < 80 before file edits.
|
|
142
|
+
- `noop`: no accepted candidates with enough evidence.
|
|
143
|
+
- `revert`: file edits were applied but validation or independent review fails.
|
|
144
|
+
|
|
145
|
+
Output format:
|
|
146
|
+
|
|
147
|
+
```json
|
|
148
|
+
{
|
|
149
|
+
"decision": "keep",
|
|
150
|
+
"score": 86,
|
|
151
|
+
"eval_mode": "independent_review",
|
|
152
|
+
"dimension_scores": {
|
|
153
|
+
"evidence_grounding": 27,
|
|
154
|
+
"project_relevance": 23,
|
|
155
|
+
"mechanical_enforceability": 12,
|
|
156
|
+
"regression_safety": 16,
|
|
157
|
+
"context_cost": 8
|
|
158
|
+
},
|
|
159
|
+
"accepted": ["quality gate requires nonzero test count"],
|
|
160
|
+
"rejected": ["generic prompt advice with no project evidence"],
|
|
161
|
+
"required_validation": ["lint-ecl", "lint-encoding", "relevant business gate"],
|
|
162
|
+
"reason": "Accepted candidate cites two archived changes and maps to the existing test command."
|
|
163
|
+
}
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
## Scoring
|
|
167
|
+
|
|
168
|
+
For each dimension, score 0-10:
|
|
169
|
+
- 10: All checks pass, high quality
|
|
170
|
+
- 7-9: Most checks pass, minor gaps
|
|
171
|
+
- 4-6: Some checks pass, significant gaps
|
|
172
|
+
- 1-3: Few checks pass, major gaps
|
|
173
|
+
- 0: Dimension entirely missing
|
|
174
|
+
|
|
175
|
+
For core-profile projects, exclude advanced-only dimensions from the weighted overall score instead
|
|
176
|
+
of scoring them as zero. For advanced-profile projects, include them and report missing directories
|
|
177
|
+
or protocols as gaps.
|
|
178
|
+
|
|
179
|
+
## Output Format
|
|
180
|
+
|
|
181
|
+
Save results to `harness/.analysis/audit.json`:
|
|
182
|
+
|
|
183
|
+
```json
|
|
184
|
+
{
|
|
185
|
+
"profile": "core",
|
|
186
|
+
"overall_score": 6.5,
|
|
187
|
+
"dimensions": {
|
|
188
|
+
"documentation": {"score": 7, "weight": 25, "checks_passed": 7, "checks_total": 9},
|
|
189
|
+
"linters": {"score": 5, "weight": 20, "checks_passed": 3, "checks_total": 6},
|
|
190
|
+
"environment": {"score": 8, "weight": 15, "checks_passed": 4, "checks_total": 5},
|
|
191
|
+
"integration": {"score": 9, "weight": 10, "checks_passed": 3, "checks_total": 3},
|
|
192
|
+
"ecl_changes": {"score": 4, "weight": 0, "checks_passed": 3, "checks_total": 7},
|
|
193
|
+
"auto_evolve": {"score": 6, "weight": 0, "checks_passed": 4, "checks_total": 7}
|
|
194
|
+
},
|
|
195
|
+
"advanced_dimensions": {
|
|
196
|
+
"evals": {"enabled": false, "reason": "advanced profile not requested"},
|
|
197
|
+
"quality_automation": {"enabled": false, "reason": "advanced profile not requested"}
|
|
198
|
+
},
|
|
199
|
+
"gaps": [
|
|
200
|
+
{"priority": "P0", "dimension": "documentation", "issue": "ARCHITECTURE.md claims 3 layers but code has 4", "fix": "Regenerate from actual imports"},
|
|
201
|
+
{"priority": "P1", "dimension": "linters", "issue": "lint-deps missing 5 packages", "fix": "Add internal/cache, internal/auth to layer map"},
|
|
202
|
+
{"priority": "P1", "dimension": "ecl_changes", "issue": "INDEX.json is hand-maintained or stale", "fix": "Generate it from archive/parking via the generated harness-change reindex command"}
|
|
203
|
+
],
|
|
204
|
+
"strengths": [
|
|
205
|
+
"Build passes cleanly",
|
|
206
|
+
"CI properly configured",
|
|
207
|
+
"Error handling is consistent"
|
|
208
|
+
]
|
|
209
|
+
}
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
Also write human-readable audit to `harness/.analysis/audit-summary.md`.
|
|
@@ -0,0 +1,343 @@
|
|
|
1
|
+
# Config & Environment Creation Agent
|
|
2
|
+
|
|
3
|
+
You are creating or updating harness configuration and environment files.
|
|
4
|
+
|
|
5
|
+
## Input
|
|
6
|
+
|
|
7
|
+
You will receive:
|
|
8
|
+
- Environment analysis (from `harness/.analysis/environment.json`)
|
|
9
|
+
- Architecture data (from `harness/.analysis/architecture.json`)
|
|
10
|
+
- Existing state (from `harness/.analysis/audit.json`)
|
|
11
|
+
- Delta list of files to create/update
|
|
12
|
+
|
|
13
|
+
## Files You Create/Update
|
|
14
|
+
|
|
15
|
+
### harness/config/environment.json
|
|
16
|
+
|
|
17
|
+
The runtime ecosystem contract. Describes what the application needs to run.
|
|
18
|
+
|
|
19
|
+
**REQUIRED FIELDS** (functional verification depends on these):
|
|
20
|
+
- `runtime.dev_command` — How to start the server in dev mode
|
|
21
|
+
- `runtime.build_command` — How to build the project
|
|
22
|
+
- `test_environment.env_vars` — Environment variables for test mode
|
|
23
|
+
- `functional_scenarios[]` — List of verification scenarios
|
|
24
|
+
|
|
25
|
+
```json
|
|
26
|
+
{
|
|
27
|
+
"runtime": {
|
|
28
|
+
"language": "go",
|
|
29
|
+
"version": "1.22",
|
|
30
|
+
"build_command": "go build ./...",
|
|
31
|
+
"dev_command": "go run main.go server -c config/server.toml",
|
|
32
|
+
"test_command": "go test ./...",
|
|
33
|
+
"binary_path": "./qts"
|
|
34
|
+
},
|
|
35
|
+
"databases": [
|
|
36
|
+
{
|
|
37
|
+
"type": "postgresql",
|
|
38
|
+
"env_vars": {"DATABASE_URL": "postgres://..."},
|
|
39
|
+
"docker": {"image": "postgres:16", "port": 5432},
|
|
40
|
+
"test_alternative": "SQLite in-memory"
|
|
41
|
+
}
|
|
42
|
+
],
|
|
43
|
+
"services": [
|
|
44
|
+
{"type": "redis", "env_vars": {"REDIS_URL": "redis://localhost:6379"}}
|
|
45
|
+
],
|
|
46
|
+
"secrets": [
|
|
47
|
+
{"name": "JWT_SECRET", "description": "JWT signing key", "test_value": "test-secret-do-not-use-in-prod"}
|
|
48
|
+
],
|
|
49
|
+
"test_environment": {
|
|
50
|
+
"env_vars": {
|
|
51
|
+
"GIN_MODE": "release",
|
|
52
|
+
"ENV_TAG": "test",
|
|
53
|
+
"LOG_LEVEL": "error"
|
|
54
|
+
}
|
|
55
|
+
},
|
|
56
|
+
"functional_scenarios": [
|
|
57
|
+
{
|
|
58
|
+
"name": "health_check",
|
|
59
|
+
"description": "Verify server starts and health endpoint responds correctly",
|
|
60
|
+
"prerequisites": ["postgresql", "redis"],
|
|
61
|
+
"steps": [
|
|
62
|
+
"Start server with runtime.dev_command",
|
|
63
|
+
"Wait for server to be ready (GET /healthz returns 200)",
|
|
64
|
+
"Verify health response contains status: up"
|
|
65
|
+
],
|
|
66
|
+
"expected_outcome": "Server is healthy and all dependencies connected"
|
|
67
|
+
},
|
|
68
|
+
{
|
|
69
|
+
"name": "basic_crud_flow",
|
|
70
|
+
"description": "Create, read, update, delete a resource via API",
|
|
71
|
+
"prerequisites": ["postgresql"],
|
|
72
|
+
"steps": [
|
|
73
|
+
"POST /api/v1/resources with valid payload -> 201",
|
|
74
|
+
"GET /api/v1/resources/:id -> 200 with matching data",
|
|
75
|
+
"PUT /api/v1/resources/:id -> 200",
|
|
76
|
+
"DELETE /api/v1/resources/:id -> 204"
|
|
77
|
+
],
|
|
78
|
+
"expected_outcome": "CRUD operations work correctly"
|
|
79
|
+
}
|
|
80
|
+
],
|
|
81
|
+
"scripts": {
|
|
82
|
+
"setup": "harness/scripts/setup-env.sh",
|
|
83
|
+
"start": "harness/scripts/start-server.sh",
|
|
84
|
+
"teardown": "harness/scripts/teardown-env.sh"
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Follow `references/environment-detection-guide.md` for detection strategies.
|
|
90
|
+
|
|
91
|
+
### harness/scripts/setup-env.sh
|
|
92
|
+
|
|
93
|
+
Start external dependencies (DB, Redis, etc.):
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
#!/bin/bash
|
|
97
|
+
set -euo pipefail
|
|
98
|
+
|
|
99
|
+
# Start PostgreSQL
|
|
100
|
+
docker run -d --name harness-postgres \
|
|
101
|
+
-p 5432:5432 \
|
|
102
|
+
-e POSTGRES_PASSWORD=testpass \
|
|
103
|
+
postgres:16
|
|
104
|
+
|
|
105
|
+
# Wait for ready
|
|
106
|
+
until docker exec harness-postgres pg_isready; do sleep 1; done
|
|
107
|
+
|
|
108
|
+
echo "✓ Environment ready"
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
If `docker-compose.yml` already exists, create a thin wrapper instead.
|
|
112
|
+
|
|
113
|
+
### harness/scripts/start-server.sh
|
|
114
|
+
|
|
115
|
+
Start the application with test environment:
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
#!/bin/bash
|
|
119
|
+
set -euo pipefail
|
|
120
|
+
|
|
121
|
+
export PORT=8081
|
|
122
|
+
export ENV=test
|
|
123
|
+
export DATABASE_URL="postgres://postgres:testpass@localhost:5432/testdb?sslmode=disable"
|
|
124
|
+
|
|
125
|
+
# Start server
|
|
126
|
+
go run cmd/api/main.go &
|
|
127
|
+
SERVER_PID=$!
|
|
128
|
+
|
|
129
|
+
# Wait for ready
|
|
130
|
+
for i in $(seq 1 30); do
|
|
131
|
+
if curl -s http://localhost:$PORT/health > /dev/null 2>&1; then
|
|
132
|
+
echo "✓ Server ready (PID: $SERVER_PID)"
|
|
133
|
+
exit 0
|
|
134
|
+
fi
|
|
135
|
+
sleep 1
|
|
136
|
+
done
|
|
137
|
+
|
|
138
|
+
echo "✗ Server failed to start"
|
|
139
|
+
exit 1
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### harness/scripts/teardown-env.sh
|
|
143
|
+
|
|
144
|
+
Stop and cleanup:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
#!/bin/bash
|
|
148
|
+
docker stop harness-postgres 2>/dev/null || true
|
|
149
|
+
docker rm harness-postgres 2>/dev/null || true
|
|
150
|
+
echo "✓ Cleaned up"
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
### Makefile Targets
|
|
154
|
+
|
|
155
|
+
Ensure these targets exist:
|
|
156
|
+
|
|
157
|
+
```makefile
|
|
158
|
+
.PHONY: lint-arch lint-ecl lint-encoding verify-harness build test setup-env start-server teardown-env
|
|
159
|
+
|
|
160
|
+
lint-arch:
|
|
161
|
+
./scripts/lint-deps
|
|
162
|
+
./scripts/lint-quality
|
|
163
|
+
|
|
164
|
+
lint-ecl:
|
|
165
|
+
{ecl_lint_command}
|
|
166
|
+
|
|
167
|
+
lint-encoding:
|
|
168
|
+
{encoding_lint_command}
|
|
169
|
+
|
|
170
|
+
verify-harness: lint-ecl lint-encoding lint-arch
|
|
171
|
+
|
|
172
|
+
build:
|
|
173
|
+
{appropriate build command}
|
|
174
|
+
|
|
175
|
+
test:
|
|
176
|
+
{appropriate test command}
|
|
177
|
+
|
|
178
|
+
setup-env:
|
|
179
|
+
./harness/scripts/setup-env.sh
|
|
180
|
+
|
|
181
|
+
start-server:
|
|
182
|
+
./harness/scripts/start-server.sh
|
|
183
|
+
|
|
184
|
+
teardown-env:
|
|
185
|
+
./harness/scripts/teardown-env.sh
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
### .github/workflows/ci.yml
|
|
189
|
+
|
|
190
|
+
Basic CI that runs build, lint, and test:
|
|
191
|
+
|
|
192
|
+
```yaml
|
|
193
|
+
name: CI
|
|
194
|
+
on: [push, pull_request]
|
|
195
|
+
jobs:
|
|
196
|
+
check:
|
|
197
|
+
runs-on: ubuntu-latest
|
|
198
|
+
steps:
|
|
199
|
+
- uses: actions/checkout@v4
|
|
200
|
+
- uses: actions/setup-{lang}@v5
|
|
201
|
+
with:
|
|
202
|
+
{lang}-version: '{version}'
|
|
203
|
+
- run: make build
|
|
204
|
+
- run: make lint-arch
|
|
205
|
+
- run: make test
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
CI must be strict by default. Include the project's normal business gates (`lint`,
|
|
209
|
+
`typecheck`, `test`, `build`, and nested package builds when detected) plus harness checks.
|
|
210
|
+
Do not remove or skip business gates because the baseline is already red; instead report those
|
|
211
|
+
failures as pre-existing project debt in the final handoff. Generate staged or relaxed CI only
|
|
212
|
+
when the user explicitly requests that tradeoff.
|
|
213
|
+
|
|
214
|
+
For TypeScript/Node.js projects, prefer package-manager scripts and Node setup over Makefile-only
|
|
215
|
+
CI. Use the adapter in `references/adapters/typescript.md` to detect npm/pnpm/yarn/bun and generate
|
|
216
|
+
commands such as `npm run lint:harness`, `npm run lint:arch`, `npm run typecheck`, `npm test`,
|
|
217
|
+
`npm run build`, and nested package build steps when present.
|
|
218
|
+
|
|
219
|
+
### Harness Directory Structure
|
|
220
|
+
|
|
221
|
+
Create the default core harness directory tree:
|
|
222
|
+
|
|
223
|
+
```
|
|
224
|
+
harness/
|
|
225
|
+
├── config/
|
|
226
|
+
│ └── environment.json
|
|
227
|
+
├── changes/
|
|
228
|
+
│ ├── active/
|
|
229
|
+
│ ├── parking/
|
|
230
|
+
│ ├── archive/
|
|
231
|
+
│ └── INDEX.json
|
|
232
|
+
├── evolution/
|
|
233
|
+
│ ├── state.json
|
|
234
|
+
│ ├── results.tsv
|
|
235
|
+
│ └── proposals/
|
|
236
|
+
├── templates/
|
|
237
|
+
│ └── change/
|
|
238
|
+
│ ├── summary.md
|
|
239
|
+
│ ├── spec.md
|
|
240
|
+
│ ├── plan.md
|
|
241
|
+
│ ├── tasks.md
|
|
242
|
+
│ └── reviews/
|
|
243
|
+
│ └── review.md
|
|
244
|
+
├── scripts/
|
|
245
|
+
│ ├── setup-env.sh
|
|
246
|
+
│ ├── start-server.sh
|
|
247
|
+
│ └── teardown-env.sh
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
Initialize `harness/evolution/state.json`:
|
|
251
|
+
|
|
252
|
+
```json
|
|
253
|
+
{
|
|
254
|
+
"enabled": true,
|
|
255
|
+
"threshold": 5,
|
|
256
|
+
"window": 10,
|
|
257
|
+
"last_evolved_archive_count": 0,
|
|
258
|
+
"last_evolved_change_id": null,
|
|
259
|
+
"last_score": null,
|
|
260
|
+
"last_run_at": null,
|
|
261
|
+
"pending": false
|
|
262
|
+
}
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
Initialize `harness/evolution/results.tsv` with this header:
|
|
266
|
+
|
|
267
|
+
```tsv
|
|
268
|
+
timestamp change_id old_score new_score status dimension note eval_mode
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
Allowed status values are `keep`, `revert`, `rejected`, and `noop`. Allowed eval modes are
|
|
272
|
+
`independent_review`, `dry_run`, and `full_test`. Use `dry_run` when no independent auditor/subagent
|
|
273
|
+
is available; do not auto-apply harness deltas in that mode.
|
|
274
|
+
|
|
275
|
+
Do not create empty advanced directories by default.
|
|
276
|
+
|
|
277
|
+
Create optional advanced directories only when the confirmed scope or user request explicitly
|
|
278
|
+
requires that capability:
|
|
279
|
+
|
|
280
|
+
```
|
|
281
|
+
harness/
|
|
282
|
+
├── eval/ # Agent evaluation datasets and runner inputs
|
|
283
|
+
├── trace/ # Agent execution traces
|
|
284
|
+
├── state/ # Runtime state for external executors
|
|
285
|
+
├── checkpoints/ # Resumable execution checkpoints
|
|
286
|
+
├── memory/ # Long-term agent memory experiments
|
|
287
|
+
│ ├── episodes/
|
|
288
|
+
│ ├── knowledge/
|
|
289
|
+
│ └── procedures/
|
|
290
|
+
└── metrics/ # Execution, quality, and cost metrics
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
Advanced directories must come with a read/write protocol and validation command. If no protocol is
|
|
294
|
+
defined, leave the capability out of the generated harness.
|
|
295
|
+
|
|
296
|
+
## Scripts Must Be
|
|
297
|
+
|
|
298
|
+
- `chmod +x` — executable
|
|
299
|
+
- Self-contained — no external dependencies beyond Docker
|
|
300
|
+
- Idempotent — safe to run multiple times
|
|
301
|
+
- With error handling — `set -euo pipefail`
|
|
302
|
+
|
|
303
|
+
## ECL Change Management Scripts
|
|
304
|
+
|
|
305
|
+
Create ECL scripts for the selected command surface from `references/ecl-harness.md`.
|
|
306
|
+
PowerShell, Bash, Node, and Python are equivalent profiles only if they implement the same commands
|
|
307
|
+
and invariants. If the project rejects `.ps1`, do not generate PowerShell as the only entrypoint.
|
|
308
|
+
For Windows projects using Bash, document Git Bash, WSL, MSYS2, or CI Linux shell as a prerequisite.
|
|
309
|
+
Select the profile automatically from project evidence; ask the user only when evidence conflicts or
|
|
310
|
+
no supported command surface is inferable.
|
|
311
|
+
|
|
312
|
+
- `scripts/harness-change.{ps1|sh|mjs|py}`: implements `new`, `status`, `validate`, `park`, `resume`, `close`, `search`, `context`, and `reindex`.
|
|
313
|
+
- `scripts/harness-evolve.{ps1|sh|mjs|py}`: implements `check`, `collect`, and `mark-complete` for default auto-evolve threshold checks.
|
|
314
|
+
- `scripts/lint-ecl.{ps1|sh|mjs|py}`: validates active change structure, `docs/STATUS.md` presence, `plan.md`, completed validation, pending task consistency, spec clarification gates, plan review gates, task id formatting, and generated index freshness.
|
|
315
|
+
- `scripts/lint-encoding.{ps1|sh|mjs|py}`: scans source/docs for mojibake markers and UTF-8 risks.
|
|
316
|
+
|
|
317
|
+
Rules:
|
|
318
|
+
- `INDEX.json` is derived from `parking/*/summary.md` and `archive/*/summary.md`; agents must not hand-edit it.
|
|
319
|
+
- `park`, `close`, `resume`, and `reindex` must rebuild `INDEX.json`.
|
|
320
|
+
- `close` and `reindex` must run `harness-evolve check`; the evolution script may create
|
|
321
|
+
`harness/evolution/pending.md`, but it must not rewrite docs, scripts, STATUS, or change files.
|
|
322
|
+
- Hook/CI integration may run validation, but must not automatically write docs, update `docs/STATUS.md`, or move changes.
|
|
323
|
+
- `harness-change context` should list active change files when present; if no active change
|
|
324
|
+
exists, it should list `harness/evolution/pending.md` before `docs/STATUS.md` when pending
|
|
325
|
+
evolution exists. It must not print or load all archive files.
|
|
326
|
+
- Auto-evolve is core lightweight infrastructure. It must not create `harness/eval`,
|
|
327
|
+
`harness/trace`, `harness/state`, `harness/checkpoints`, `harness/memory`, or
|
|
328
|
+
`harness/metrics` unless the user explicitly requested those advanced capabilities.
|
|
329
|
+
- Wire Makefile/package scripts/CI to the selected entrypoint, for example
|
|
330
|
+
`bash scripts/lint-ecl.sh` for Bash or `{pkg_manager} run lint:harness` for package scripts.
|
|
331
|
+
- If the PowerShell profile is selected, detect whether `pwsh` exists before documenting or wiring
|
|
332
|
+
commands. If `pwsh` is unavailable, use `powershell -NoProfile -ExecutionPolicy Bypass`.
|
|
333
|
+
- Keep PowerShell profile scripts compatible with Windows PowerShell 5.1. Avoid ambiguous overloads such as
|
|
334
|
+
`TrimStart(".\")`; use typed arguments such as `[char[]]@(".", "\")`.
|
|
335
|
+
- Avoid non-ASCII mojibake marker literals in PowerShell templates. Use Unicode codepoints or another
|
|
336
|
+
PowerShell 5.1-safe representation so the script still parses on legacy Windows code pages.
|
|
337
|
+
|
|
338
|
+
## Verification Config Rule
|
|
339
|
+
|
|
340
|
+
`harness/config/environment.json` is the static runtime contract created by ecl-harness-engineer.
|
|
341
|
+
Do not create `harness/config/verify.json` or `harness/config/validate.json`; task-specific
|
|
342
|
+
verification plans are generated later by the executor/runtime from `environment.json` plus
|
|
343
|
+
the active change context.
|