harness-everything 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +2 -0
- package/BENCHMARK_SOP.md +62 -0
- package/CHANGELOG.md +107 -0
- package/Dockerfile +18 -0
- package/LICENSE +201 -0
- package/README.md +197 -0
- package/VERIFICATION.md +347 -0
- package/bin/cli.js +97 -0
- package/build-multi-agent-system/AGENTS.md +50 -0
- package/build-multi-agent-system/README.md +63 -0
- package/build-multi-agent-system/SKILL.md +68 -0
- package/create-agent-launcher/SKILL.md +48 -0
- package/create-agent-launcher/examples/csharp-monorepo.md +123 -0
- package/create-agent-launcher/examples/typescript-cli.md +126 -0
- package/create-agent-launcher/guidelines/platform-claude.md +77 -0
- package/create-agent-launcher/guidelines/platform-codex.md +146 -0
- package/create-agent-launcher/guidelines/platform-copilot.md +74 -0
- package/create-agent-launcher/guidelines/platform-gemini.md +101 -0
- package/create-agent-launcher/guidelines/role-capabilities.md +108 -0
- package/create-agent-launcher/templates/README.md +17 -0
- package/create-agent-launcher/templates/agent-launcher.md +68 -0
- package/create-agent-launcher/templates/backend-developer-csharp.md +56 -0
- package/create-agent-launcher/templates/backend-developer-polyglot.md +62 -0
- package/create-agent-launcher/templates/backend-developer-ts.md +57 -0
- package/create-agent-launcher/templates/memory-keeper.md +37 -0
- package/create-agent-launcher/templates/requirement-analyzer.md +66 -0
- package/create-agent-launcher/templates/specialist-agent.md +34 -0
- package/create-agent-launcher/templates/task-verifier.md +56 -0
- package/create-agent-launcher/templates/tool-maker.md +33 -0
- package/create-agent-launcher/workflows/01-init.md +22 -0
- package/create-agent-launcher/workflows/02-analysis.md +26 -0
- package/create-agent-launcher/workflows/03-generation.md +29 -0
- package/create-agent-launcher/workflows/04-launcher.md +20 -0
- package/docker-verify.sh +266 -0
- package/docs/architecture.md +95 -0
- package/docs/philosophy.md +45 -0
- package/docs/reflection.md +34 -0
- package/docs/routing.md +45 -0
- package/docs/workflows/build-multi-agent-system.md +62 -0
- package/docs/workflows/create-agent-launcher.md +62 -0
- package/docs/workflows/environment-detection.md +63 -0
- package/docs/workflows/eval-harness.md +57 -0
- package/docs/workflows/fable-discipline.md +59 -0
- package/docs/workflows/fable-mode.md +32 -0
- package/docs/workflows/git-commit.md +62 -0
- package/docs/workflows/grill-me.md +62 -0
- package/docs/workflows/grill-with-docs.md +62 -0
- package/docs/workflows/harness-everything.md +92 -0
- package/docs/workflows/improve-codebase-architecture.md +60 -0
- package/docs/workflows/install-cognitive-os.md +60 -0
- package/docs/workflows/repo-docs.md +59 -0
- package/docs/workflows/rewrite-commits.md +60 -0
- package/docs/workflows/security-review.md +58 -0
- package/docs/workflows/self-evolve.md +59 -0
- package/docs/workflows/skill-creator.md +75 -0
- package/docs/workflows/skill-style.md +64 -0
- package/docs/workflows/tdd.md +39 -0
- package/docs/workflows/todo-driven-workflow.md +36 -0
- package/docs/workflows/using-git-worktrees.md +63 -0
- package/docs/workflows/verification-loop.md +69 -0
- package/docs/workflows/verify-before-claim.md +60 -0
- package/docs/workflows/zoom-out.md +62 -0
- package/environment-detection/SKILL.md +77 -0
- package/environment-detection/scripts/preflight.js +93 -0
- package/eval-framework/behavioral-test.js +105 -0
- package/eval-framework/cases/case1.json +5 -0
- package/eval-framework/cases/case2.json +5 -0
- package/eval-framework/cases/case3.json +5 -0
- package/eval-framework/cases/case4-zh-tier2.json +5 -0
- package/eval-framework/cases/case5-zh-tier3.json +5 -0
- package/eval-framework/mechanism-test.js +251 -0
- package/eval-framework/runner.js +56 -0
- package/eval-harness/SKILL.md +59 -0
- package/eval-harness/scripts/evaluate.js +61 -0
- package/fable-discipline/SKILL.md +38 -0
- package/fable-mode/CONTRACT-FORMAT.md +51 -0
- package/fable-mode/SKILL.md +51 -0
- package/fable-mode/agents/fable-orchestrator.md +92 -0
- package/fable-mode/agents/fable-verifier.md +32 -0
- package/fable-mode/agents/fable-worker-haiku.md +48 -0
- package/fable-mode/agents/fable-worker-sonnet.md +49 -0
- package/fable-mode/execution-guardrails/SKILL.md +87 -0
- package/fable-mode/fable-haiku/SKILL.md +48 -0
- package/git-commit/SKILL.md +55 -0
- package/git-commit/guides/ANGULAR_STYLE.md +61 -0
- package/git-commit/guides/COMMIT_GENERATION.md +24 -0
- package/git-commit/guides/LANGUAGE_DETECTION.md +16 -0
- package/git-commit/guides/MAIN_REPO.md +15 -0
- package/git-commit/guides/SUBMODULES.md +27 -0
- package/grill-me/SKILL.md +36 -0
- package/grill-with-docs/ADR-FORMAT.md +47 -0
- package/grill-with-docs/CONTEXT-FORMAT.md +63 -0
- package/grill-with-docs/SKILL.md +99 -0
- package/harness-everything/SKILL.md +141 -0
- package/harness-everything/examples/README.md +9 -0
- package/harness-everything/examples/todo-workflow.md +22 -0
- package/harness-everything/scripts/bootstrap.js +140 -0
- package/harness-everything/scripts/self-heal.js +189 -0
- package/harness-everything/scripts/tier-router.js +219 -0
- package/harness-everything/scripts/todo-cli.js +157 -0
- package/harness-everything/scripts/verify-gate.js +31 -0
- package/hooks/hooks.json +155 -0
- package/hooks/scripts/atomic-commit-check.js +68 -0
- package/hooks/scripts/boundary-guard.js +55 -0
- package/hooks/scripts/context-compact.js +114 -0
- package/hooks/scripts/contract-test.js +79 -0
- package/hooks/scripts/depth-guard.js +65 -0
- package/hooks/scripts/lib/harness-state.js +202 -0
- package/hooks/scripts/lib/platforms/claude.js +42 -0
- package/hooks/scripts/lib/platforms/codex.js +46 -0
- package/hooks/scripts/lib/platforms/continue.js +41 -0
- package/hooks/scripts/lib/platforms/copilot.js +41 -0
- package/hooks/scripts/lib/platforms/cursor.js +41 -0
- package/hooks/scripts/lib/platforms/hermes.js +32 -0
- package/hooks/scripts/lib/platforms/index.js +17 -0
- package/hooks/scripts/lib/platforms/worktrees.js +20 -0
- package/hooks/scripts/reset-circuit-breaker.js +30 -0
- package/hooks/scripts/rule-of-3-tracker.js +91 -0
- package/hooks/scripts/rule-of-3.js +176 -0
- package/hooks/scripts/state-persist.js +99 -0
- package/hooks/scripts/stop-gate.js +80 -0
- package/hooks/scripts/subagent-scope-guard.js +88 -0
- package/improve-codebase-architecture/SKILL.md +44 -0
- package/improve-codebase-architecture/guides/DEEPENING.md +37 -0
- package/improve-codebase-architecture/guides/HTML-REPORT.md +123 -0
- package/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -0
- package/improve-codebase-architecture/guides/LANGUAGE.md +53 -0
- package/install-cognitive-os/SKILL.md +95 -0
- package/package.json +22 -0
- package/repo-docs/SKILL.md +45 -0
- package/repo-docs/templates/agents-template.md +264 -0
- package/repo-docs/templates/knowledge-agents-template.md +50 -0
- package/repo-docs/templates/knowledge-readme-template.md +83 -0
- package/repo-docs/templates/manual-recon.md +117 -0
- package/repo-docs/templates/multi-skills-readme-template.md +72 -0
- package/repo-docs/templates/product-agents-template.md +78 -0
- package/repo-docs/templates/product-readme-template.md +95 -0
- package/repo-docs/templates/readme-template.md +136 -0
- package/repo-docs/templates/single-skill-readme-template.md +70 -0
- package/repo-docs/templates/skills-agents-template.md +60 -0
- package/rewrite-commits/SKILL.md +32 -0
- package/scripts/installer.js +649 -0
- package/scripts/lib/advisory-text.js +163 -0
- package/scripts/lib/claude-hooks.js +71 -0
- package/scripts/lib/gitignore.js +58 -0
- package/scripts/lib/manifest.js +82 -0
- package/scripts/lib/prompts.js +241 -0
- package/scripts/lib/skills.js +172 -0
- package/scripts/lib/workspace.js +28 -0
- package/security-review/SKILL.md +511 -0
- package/self-evolve/SKILL.md +54 -0
- package/self-evolve/scripts/persist-memory.js +40 -0
- package/self-evolve/scripts/self-regression.js +113 -0
- package/skill-creator/SKILL.md +91 -0
- package/skill-creator/references/quality-principles.md +81 -0
- package/skill-creator/references/testing-workflow.md +32 -0
- package/skill-style/SKILL.md +48 -0
- package/skill-style/STYLE.md +37 -0
- package/tdd/SKILL.md +53 -0
- package/tdd/guides/deep-modules.md +33 -0
- package/tdd/guides/interface-design.md +31 -0
- package/tdd/guides/mocking.md +59 -0
- package/tdd/guides/refactoring.md +10 -0
- package/tdd/guides/tests.md +61 -0
- package/todo-driven-workflow/SKILL.md +54 -0
- package/using-git-worktrees/SKILL.md +161 -0
- package/verification-loop/SKILL.md +138 -0
- package/verify-before-claim/SKILL.md +86 -0
- package/zoom-out/SKILL.md +90 -0
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: self-evolve
|
|
3
|
+
description: Extracts root causes from resolved difficult problems and persists them as new error boundaries in memory.
|
|
4
|
+
author: Miya Daniel | Harness Core Team
|
|
5
|
+
version: 0.2.0
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Self Evolve (Self Evolution & Memory Extraction)
|
|
9
|
+
|
|
10
|
+
## π Skill Contract
|
|
11
|
+
|
|
12
|
+
| Component | Specification |
|
|
13
|
+
| :--- | :--- |
|
|
14
|
+
| **Trigger / Input** | Task completion after a major struggle, or post-zoom-out recovery. Input: The root cause of the previous failure. |
|
|
15
|
+
| **Expected Output** | Terminal execution of the persistence script. |
|
|
16
|
+
| **State Mutations** | Updates workspace memory/rules files (e.g., `RULES.md` or `.claude/harness-state/memories.json`). |
|
|
17
|
+
| **Enforcement Gate** | You MUST run `node <this-skill-dir>/scripts/self-regression.js`. You MUST run `node <this-skill-dir>/scripts/persist-memory.js "<rule>"`. |
|
|
18
|
+
|
|
19
|
+
This skill is responsible for the long-term learning and error prevention of the system.
|
|
20
|
+
It ensures that the entire Harness ecosystem becomes smarter and avoids repeating mistakes after solving difficult problems every time.
|
|
21
|
+
|
|
22
|
+
## 1. Triggers
|
|
23
|
+
- **Post-Circuit Breaker**: When `zoom-out` was triggered, and the difficult problem was ultimately solved β via the reflection report's fresh diagnosis or with the human's hints.
|
|
24
|
+
- **Major Breakthrough**: Completed an extremely complex Tier 3 task, and discovered some previously unknown framework limitations or architectural pitfalls during the process.
|
|
25
|
+
- **Human Instruction**: The user explicitly requests "remember this lesson" or "add this to memory".
|
|
26
|
+
|
|
27
|
+
## 2. Execution Process (The Evolution Loop)
|
|
28
|
+
|
|
29
|
+
### Step 1: Deep Reflection & Root Cause Extraction
|
|
30
|
+
- **Reflection is Core**: Reflection is the true focus of the memory system. Mere recording cannot improve accuracy; you must continuously reflect to self-evolve.
|
|
31
|
+
- Analyze the problem just solved, discarding specific variable names or specific line numbers of code.
|
|
32
|
+
- Extract the **high-level, universally applicable root cause**.
|
|
33
|
+
- *Incorrect Extraction*: "Do not use `forEach` to call the database on line 45 of `auth.js`."
|
|
34
|
+
- *Correct Extraction*: "When handling ORM relational writes in this project, you MUST use Transactions and batch writes. Sending Queries individually in a loop is PROHIBITED, otherwise it will cause connection pool exhaustion."
|
|
35
|
+
|
|
36
|
+
### Step 2: Define Error Boundaries
|
|
37
|
+
- Transform the extracted root cause into a defensive rule for `install-cognitive-os` to reference in the future.
|
|
38
|
+
- Explicitly define which domain this new rule applies to (e.g., Database, React Hooks, specific API integration).
|
|
39
|
+
- **Anti-Focus Loss**: This rule must be able to directly guide future Agents to notice this landmine during the `[Discover]` phase, instead of finding out only after writing it wrong.
|
|
40
|
+
|
|
41
|
+
### Step 3: Memory Persistence & Dynamic Skill Generation
|
|
42
|
+
- **MANDATORY**: Before persisting or committing any dynamically generated skills or memories, you MUST run the self-regression suite to ensure no behavioral regressions or syntax errors were introduced. Both scripts below live in the `scripts/` directory **inside this skill's own directory** β resolve paths from wherever this SKILL.md was loaded (do not guess a hard-coded location):
|
|
43
|
+
```bash
|
|
44
|
+
node "<this-skill-dir>/scripts/self-regression.js"
|
|
45
|
+
```
|
|
46
|
+
- **MANDATORY**: You MUST execute the persistence script instead of manually editing files or calling write tools:
|
|
47
|
+
```bash
|
|
48
|
+
node "<this-skill-dir>/scripts/persist-memory.js" "<Your extracted root cause and defensive rule here>"
|
|
49
|
+
```
|
|
50
|
+
- **Dynamic Skill Generation (Session Packaging)**: Through self-review, package the successful session summary (hard boundary) into a new, dynamic skill. **You MUST load `skill-creator/SKILL.md` and follow its Β§4 Dynamic Skill Generation Contract** before writing the file β it defines the required location (`.claude/harness-everything/skills/generated/<name>/SKILL.md`, not the repo root), the required lifecycle frontmatter (`metadata.type: dynamic`, `status: draft`), and the Quality Checklist gate this file must pass since it skips human PR review by design. Do not hand-write a `SKILL.md` inline here; that checklist is the only review a dynamically generated skill gets. These dynamically generated skills have a lifecycle β draft β active β deprecated, or promoted to a static skill once proven general β distinct from the system's foundational static skills.
|
|
51
|
+
- **Note**: Only record "Key Insights", keeping the memory document short and punchy. Avoid stuffing it with lengthy conversation logs or useless narratives.
|
|
52
|
+
|
|
53
|
+
## 3. Purpose
|
|
54
|
+
Through `self-evolve`, we transform "invalid trial-and-error", which might otherwise waste Tokens, into a valuable "moat" for the system. Even if the underlying model doesn't become inherently smarter, equipped with these memories, the system will automatically avoid known traps and break through its original reasoning ceiling.
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
const fs = require('fs');
|
|
3
|
+
const path = require('path');
|
|
4
|
+
|
|
5
|
+
const args = process.argv.slice(2);
|
|
6
|
+
const memoryText = args[0];
|
|
7
|
+
|
|
8
|
+
if (!memoryText) {
|
|
9
|
+
console.error("Usage: node persist-memory.js \"<insight to remember>\"");
|
|
10
|
+
process.exit(1);
|
|
11
|
+
}
|
|
12
|
+
|
|
13
|
+
function getWorkspaceRoot() {
|
|
14
|
+
let dir = path.resolve(process.cwd());
|
|
15
|
+
while (dir !== path.parse(dir).root) {
|
|
16
|
+
if (fs.existsSync(path.join(dir, '.git'))) return dir;
|
|
17
|
+
dir = path.dirname(dir);
|
|
18
|
+
}
|
|
19
|
+
return process.cwd();
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
// Ensure the memory is written to the project root's /memories/repo/ directory
|
|
23
|
+
const currentDir = getWorkspaceRoot();
|
|
24
|
+
const memoryDir = path.join(currentDir, 'memories', 'repo');
|
|
25
|
+
const rulesFile = path.join(memoryDir, 'RULES.md');
|
|
26
|
+
|
|
27
|
+
try {
|
|
28
|
+
if (!fs.existsSync(memoryDir)) {
|
|
29
|
+
fs.mkdirSync(memoryDir, { recursive: true });
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
const timestamp = new Date().toISOString().split('T')[0];
|
|
33
|
+
const formattedMemory = `\n## [${timestamp}] Self-Evolution Insight\n- ${memoryText}\n`;
|
|
34
|
+
|
|
35
|
+
fs.appendFileSync(rulesFile, formattedMemory, 'utf8');
|
|
36
|
+
console.log(`[Success] Memory persisted to ${rulesFile}`);
|
|
37
|
+
} catch (err) {
|
|
38
|
+
console.error(`[Error] Failed to write memory: ${err.message}`);
|
|
39
|
+
process.exit(1);
|
|
40
|
+
}
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Harness Self-Regression Test Suite
|
|
4
|
+
* Compiles and validates all Javascript files and runs routing verification framework
|
|
5
|
+
* before dynamic evolution / dynamic skills are officially persisted.
|
|
6
|
+
*/
|
|
7
|
+
|
|
8
|
+
const { spawnSync } = require('child_process');
|
|
9
|
+
const path = require('path');
|
|
10
|
+
const fs = require('fs');
|
|
11
|
+
|
|
12
|
+
const projectRoot = path.resolve(__dirname, '..', '..');
|
|
13
|
+
|
|
14
|
+
console.log("=================================================");
|
|
15
|
+
console.log(" Harness OS - Self-Regression Test Suite ");
|
|
16
|
+
console.log("=================================================");
|
|
17
|
+
|
|
18
|
+
let hasErrors = false;
|
|
19
|
+
|
|
20
|
+
// 1. Walk and syntax check all Javascript files
|
|
21
|
+
console.log("\n[Phase 1] Static Syntax Check...");
|
|
22
|
+
const foldersToScan = ['harness-everything', 'hooks', 'environment-detection', 'self-evolve', 'eval-framework', 'eval-harness', 'scripts', 'bin'];
|
|
23
|
+
const jsFiles = [];
|
|
24
|
+
|
|
25
|
+
function walkDir(dir) {
|
|
26
|
+
if (!fs.existsSync(dir)) return;
|
|
27
|
+
const list = fs.readdirSync(dir);
|
|
28
|
+
list.forEach(file => {
|
|
29
|
+
const filePath = path.join(dir, file);
|
|
30
|
+
const stat = fs.statSync(filePath);
|
|
31
|
+
if (stat && stat.isDirectory()) {
|
|
32
|
+
walkDir(filePath);
|
|
33
|
+
} else if (filePath.endsWith('.js')) {
|
|
34
|
+
jsFiles.push(filePath);
|
|
35
|
+
}
|
|
36
|
+
});
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
foldersToScan.forEach(folder => {
|
|
40
|
+
walkDir(path.join(projectRoot, folder));
|
|
41
|
+
});
|
|
42
|
+
|
|
43
|
+
jsFiles.forEach(file => {
|
|
44
|
+
const relativePath = path.relative(projectRoot, file);
|
|
45
|
+
// node --check checks syntax without execution
|
|
46
|
+
const check = spawnSync('node', ['--check', file]);
|
|
47
|
+
if (check.status !== 0) {
|
|
48
|
+
console.error(` β Syntax Error in ${relativePath}`);
|
|
49
|
+
console.error(check.stderr.toString().trim());
|
|
50
|
+
hasErrors = true;
|
|
51
|
+
} else {
|
|
52
|
+
console.log(` β
${relativePath.padEnd(50)} [Valid Syntax]`);
|
|
53
|
+
}
|
|
54
|
+
});
|
|
55
|
+
|
|
56
|
+
// 2. Run Tier Verification Framework
|
|
57
|
+
console.log("\n[Phase 2] Routing Verification Check...");
|
|
58
|
+
const runnerPath = path.join(projectRoot, 'eval-framework', 'runner.js');
|
|
59
|
+
if (fs.existsSync(runnerPath)) {
|
|
60
|
+
const runnerCheck = spawnSync('node', [runnerPath], { stdio: 'inherit' });
|
|
61
|
+
if (runnerCheck.status !== 0) {
|
|
62
|
+
console.error("\n β Routing verification framework failed!");
|
|
63
|
+
hasErrors = true;
|
|
64
|
+
} else {
|
|
65
|
+
console.log("\n β
Routing verification framework 100% Passed!");
|
|
66
|
+
}
|
|
67
|
+
} else {
|
|
68
|
+
console.warn(" β οΈ eval-framework/runner.js not found. Skipping Phase 2.");
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
// 3. Run Behavioral Test Suite (E2E state-machine transitions, not just syntax)
|
|
72
|
+
console.log("\n[Phase 3] Behavioral Test Suite...");
|
|
73
|
+
const behavioralTestPath = path.join(projectRoot, 'eval-framework', 'behavioral-test.js');
|
|
74
|
+
if (fs.existsSync(behavioralTestPath)) {
|
|
75
|
+
const behavioralCheck = spawnSync('node', [behavioralTestPath], { stdio: 'inherit', cwd: projectRoot });
|
|
76
|
+
if (behavioralCheck.status !== 0) {
|
|
77
|
+
console.error("\n β Behavioral test suite failed!");
|
|
78
|
+
hasErrors = true;
|
|
79
|
+
} else {
|
|
80
|
+
console.log("\n β
Behavioral test suite 100% Passed!");
|
|
81
|
+
}
|
|
82
|
+
} else {
|
|
83
|
+
console.warn(" β οΈ eval-framework/behavioral-test.js not found. Skipping Phase 3.");
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
// 4. Run Mechanism Test Suite (VERIFICATION.md Β§2 hook contract, automated)
|
|
87
|
+
console.log("\n[Phase 4] Mechanism Test Suite (Claude Code hooks)...");
|
|
88
|
+
const mechanismTestPath = path.join(projectRoot, 'eval-framework', 'mechanism-test.js');
|
|
89
|
+
if (fs.existsSync(mechanismTestPath)) {
|
|
90
|
+
const mechanismCheck = spawnSync('node', [mechanismTestPath], { stdio: 'inherit', cwd: projectRoot });
|
|
91
|
+
if (mechanismCheck.status !== 0) {
|
|
92
|
+
console.error("\n β Mechanism test suite failed!");
|
|
93
|
+
hasErrors = true;
|
|
94
|
+
} else {
|
|
95
|
+
console.log("\n β
Mechanism test suite 100% Passed!");
|
|
96
|
+
}
|
|
97
|
+
} else {
|
|
98
|
+
console.warn(" β οΈ eval-framework/mechanism-test.js not found. Skipping Phase 4.");
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
// 5. Final verdict
|
|
102
|
+
console.log("\n=================================================");
|
|
103
|
+
if (hasErrors) {
|
|
104
|
+
console.error(" β REGRESSION DETECTED! Self-evolution rejected.");
|
|
105
|
+
console.error(" Please fix the errors listed above before persisting changes.");
|
|
106
|
+
console.log("=================================================");
|
|
107
|
+
process.exit(1);
|
|
108
|
+
} else {
|
|
109
|
+
console.log(" π SUCCESS! All self-regression checks passed 100%.");
|
|
110
|
+
console.log(" Dynamic skills and memories are safe to be committed.");
|
|
111
|
+
console.log("=================================================");
|
|
112
|
+
process.exit(0);
|
|
113
|
+
}
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: skill-creator
|
|
3
|
+
description: Author, audit, and refactor SKILL.md files against a single quality bar β the Skill Contract format plus predictability/pruning/progressive-disclosure principles from external skill-writing research. Use when creating a new skill from scratch, reviewing or refactoring an existing SKILL.md, checking whether two skills overlap, or when self-evolve needs to package a session's learning into a durable dynamic skill.
|
|
4
|
+
author: Miya Daniel | Harness Core Team
|
|
5
|
+
version: 0.2.0
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Skill Creator (Skill Authoring, Audit & Evolution Workflow)
|
|
9
|
+
|
|
10
|
+
## π Skill Contract
|
|
11
|
+
|
|
12
|
+
| Component | Specification |
|
|
13
|
+
| :--- | :--- |
|
|
14
|
+
| **Trigger / Input** | Creating a new skill from scratch; auditing/refactoring an existing `SKILL.md`; `self-evolve`'s Step 3 packaging a session's insight into a durable dynamic skill. Input: a task description, an existing SKILL.md, or a root-cause insight from self-evolve. |
|
|
15
|
+
| **Expected Output** | A `SKILL.md` that passes the Quality Checklist (Β§3) β static skills at `<skill-name>/SKILL.md` with a `harness-everything` registry row, dynamic skills at `.harness/skills/generated/<skill-name>/SKILL.md` with lifecycle metadata (Β§4). |
|
|
16
|
+
| **State Mutations** | Writes/edits `<skill>/SKILL.md` (+ optional `guides/`/`references/`); for new static skills, adds one row to `harness-everything/SKILL.md` Β§5 and, if keyword-routable, one line to `tier-router.js`'s `recommendedGuides`. |
|
|
17
|
+
| **Enforcement Gate** | A new or edited `SKILL.md` **MUST NOT** be registered or persisted until every item in the Quality Checklist (Β§3) is checked off β for dynamic skills this is self-evolve's actual persistence gate, not optional polish. |
|
|
18
|
+
|
|
19
|
+
`skill-style/SKILL.md` defines *the shape* a SKILL.md must take (the Skill Contract table, the imperative tone) but not *how to get there well* β how to size a skill, when to push detail into a reference file, how to tell a real enforcement gate from a restated default, or where a rule belongs before it starts drifting across three files. Those questions come from two sources studied for this skill: [`writing-great-skills`](../mattpocock-skills/skills/productivity/writing-great-skills/SKILL.md) and Anthropic's own `skill-creator`. Neither is followed blindly β Β§1 says where Harness's own conventions override them, and why.
|
|
20
|
+
|
|
21
|
+
## 1. Two philosophies, reconciled
|
|
22
|
+
|
|
23
|
+
Harness gets predictability from a **physical** mechanism: the Skill Contract table's Enforcement Gate row, backed by a script that exits non-zero. External skill-writing research gets predictability from a **cognitive** mechanism: minimal context load, leading words, explaining *why* instead of just forbidding. Both are real and both are validated β see the 2026-07-22 skill quality audit for the evidence β and they don't actually conflict where it counts:
|
|
24
|
+
|
|
25
|
+
- **Keep** the Skill Contract table and `MUST`/`PROHIBITED` for anything with a real Enforcement Gate: an irreversible action, a script that can exit non-zero, a state mutation another skill depends on. That's a hard guardrail; state it as one.
|
|
26
|
+
- **Don't** reach for `MUST`/`PROHIBITED` as the default tone for style preferences with no gate behind them β "don't write long paragraphs" doesn't need to be a `MUST NOT`; say what good output looks like instead. Every imperative not backed by an actual gate is a candidate for rewriting positively.
|
|
27
|
+
- **Always** apply the parts with no philosophical conflict at all: don't duplicate a meaning across files, don't restate what the model already does by default, push detail behind a pointer once a skill grows past what every reader needs inline, and reuse this repo's own leading words (`Tier 1/2/3`, `circuit breaker`/`zoom out`, `Rule of 3`, `Red/Green/Refactor`) instead of coining new ones for the same concept.
|
|
28
|
+
|
|
29
|
+
Full glossary and worked examples from both sources: `references/quality-principles.md`.
|
|
30
|
+
|
|
31
|
+
## 2. Authoring workflow
|
|
32
|
+
|
|
33
|
+
### Step 1 β Capture intent (before writing a word of SKILL.md)
|
|
34
|
+
Answer these, pulling from the conversation first if it already has the answers:
|
|
35
|
+
1. What should this skill make the agent do that it wouldn't do by default?
|
|
36
|
+
2. What's the *one* canonical sentence describing when it fires? This sentence becomes the frontmatter `description` β and nothing else. Don't let the registry row or `tier-router.js` invent independent wording later: the audit report Β§1.2 found the same skill carrying two different one-line descriptions inside `tier-router.js` itself, because nobody treated the frontmatter line as the source of truth.
|
|
37
|
+
3. Which Tier does it belong to, or is it an always-on discipline / meta skill outside the tier system?
|
|
38
|
+
4. Does an existing skill already own this ground? Grep `harness-everything/SKILL.md` Β§5 first β a near-duplicate skill is worse than a slightly-too-broad existing one.
|
|
39
|
+
|
|
40
|
+
### Step 2 β Draft
|
|
41
|
+
Write the Skill Contract table first β it forces you to know the Enforcement Gate before you write a paragraph. Then the body, sized by the information hierarchy:
|
|
42
|
+
- **Steps** (ordered, each ending on a checkable completion criterion) for anything the agent *does*, in order.
|
|
43
|
+
- **Flat reference** (a rule table, a decision list) for anything the agent *consults*. A skill can be all steps, all reference, or both β a flat rule table is a legitimate shape, not a smell.
|
|
44
|
+
- Past ~100-150 lines of body, or once a piece of detail is only needed on some branches rather than every invocation, push it to `guides/` or `references/` with a pointer sentence that says *when* to open it, not a bare link. Past 300 lines in a reference file, add a table of contents. `git-commit`, `tdd`, `repo-docs`, and `improve-codebase-architecture` already do this well β copy their shape.
|
|
45
|
+
|
|
46
|
+
### Step 3 β Test against real prompts
|
|
47
|
+
Don't ship on vibes alone. Write 2-3 prompts a real user would actually type β not "test the skill," the kind of message that should trigger it. Using this repo's own `create-agent-launcher`, spawn one subagent *with* the draft skill loaded and one *without* (baseline), same prompt, and compare. This is the same with/without comparison Anthropic's `skill-creator` runs through a Python eval-viewer β Harness doesn't need a second toolchain for it; `create-agent-launcher` already does resource-isolated subagent spawning. Read both transcripts, not just final outputs β repeated dead-end exploration in the with-skill transcript is a sign the skill under-specifies something, not that the model is weak.
|
|
48
|
+
|
|
49
|
+
Full adapted testing loop β when to bother, what to look for in transcripts, how a router-based system checks trigger phrasing without a native autonomous picker: `references/testing-workflow.md`.
|
|
50
|
+
|
|
51
|
+
### Step 4 β Prune, then register
|
|
52
|
+
Run the Quality Checklist (Β§3). Then:
|
|
53
|
+
- **Static skill**: add one row to `harness-everything/SKILL.md` Β§5 (Layer + "Activated when," quoting the frontmatter description rather than rewriting it), and if the trigger is keyword-detectable, one line to the matching keyword block in `harness-everything/scripts/tier-router.js`.
|
|
54
|
+
- **Dynamic skill** (from `self-evolve`): follow Β§4 instead β different location, different lifecycle, same checklist.
|
|
55
|
+
|
|
56
|
+
## 3. Quality Checklist (the actual gate)
|
|
57
|
+
|
|
58
|
+
A skill isn't done until every line here is true. This is deliberately a flat checklist, not steps β check them in any order, but check all of them.
|
|
59
|
+
|
|
60
|
+
- [ ] **Skill Contract table present and accurate** β all four rows; Enforcement Gate names a real script/exit-code or an explicit "None" if the skill is pure reference.
|
|
61
|
+
- [ ] **Single source of truth**: the frontmatter `description` is the *only* place this skill's purpose is originally written. The registry row and any `tier-router.js` line quote or tightly paraphrase it β they don't independently re-describe the skill.
|
|
62
|
+
- [ ] **No block over ~5 lines duplicated verbatim elsewhere** β in this file or in another always/near-always-loaded skill. (`install-cognitive-os` vs. `harness-everything`'s 33-line ADHD-output block, flagged in the audit report, is the canonical counter-example β don't add a second one.)
|
|
63
|
+
- [ ] **Every `MUST`/`MUST NOT`/`PROHIBITED` has a real gate behind it**, or is rewritten as a positive statement of the target behavior. A prohibition with no gate and no "why" sentence next to it is a smell, not a strength.
|
|
64
|
+
- [ ] **No no-op lines** β for each imperative sentence, ask whether it changes behavior versus what the model already does by default. ("Be thorough" reads as a no-op; a concrete completion criterion or a stronger leading word doesn't.)
|
|
65
|
+
- [ ] **Body fits the information hierarchy** β steps for ordered action, flat reference for rules, detail past what every branch needs pushed to `guides/`/`references/` with a "when to open this" pointer.
|
|
66
|
+
- [ ] **Reuses existing leading words** (`Tier 1/2/3`, `circuit breaker`/`zoom out`, `Rule of 3`, `Red/Green/Refactor`, `Discover > Think > Try > Summarize > Record`) instead of naming a new concept for something this repo already has a word for.
|
|
67
|
+
- [ ] **Completion criteria are checkable** β tied to an exit code, a file's existence, a grep result, or explicit human confirmation, not "when it feels done."
|
|
68
|
+
- [ ] **Description reads as a human-facing sentence**, not just a trigger-phrase list β `scripts/installer.js`'s interactive picker prints it next to a checkbox for a person deciding whether to install this skill; that's its real first reader in this architecture (audit report Β§2.1).
|
|
69
|
+
|
|
70
|
+
## 4. Dynamic Skill Generation Contract (for `self-evolve`)
|
|
71
|
+
|
|
72
|
+
`self-evolve`'s Step 3 packages a hard-won session insight into a new, standalone skill. That new skill **MUST** go through this skill, not be hand-written inline:
|
|
73
|
+
|
|
74
|
+
- **Location**: `.claude/harness-everything/skills/generated/<kebab-case-name>/SKILL.md` β not the repo root. `.claude/harness-everything/skills` is already a scope the installer recognizes (`scripts/lib/skills.js`'s `getInstalledSkills`), which keeps dynamically-generated skills discoverable without mixing them into the reviewed, static skill set at the repo root. `generated/` is deliberately excluded from the installer's manifest-tracked, bulk-removable skill set β these are the user's own accumulated learning, not package content, so "uninstall skills" must never sweep them.
|
|
75
|
+
- **Required frontmatter**, in addition to `name`/`description`:
|
|
76
|
+
```yaml
|
|
77
|
+
metadata:
|
|
78
|
+
type: dynamic
|
|
79
|
+
generated: <YYYY-MM-DD>
|
|
80
|
+
source: <one-line pointer to the session/root-cause that produced it, e.g. "zoom-out recovery, 2026-07-22, ORM connection-pool exhaustion">
|
|
81
|
+
status: draft
|
|
82
|
+
```
|
|
83
|
+
- **Gate**: run the Quality Checklist (Β§3) before the skill file is written. A dynamic skill skips human PR review by design, so this checklist is the only review it gets β don't skip it because the insight "feels obviously right" in the moment; that's exactly the state self-evolve is triggered from.
|
|
84
|
+
- **Lifecycle**: `status: draft` at birth. Promote to `status: active` once it fires successfully in a *different* task than the one that produced it β a skill that has only proven itself on the bug that spawned it hasn't proven generality yet. If a dynamic skill goes unused or stops matching reality, mark `status: deprecated` rather than deleting it silently β deleting is fine once something else supersedes it, but a silent disappearance is harder to debug later than a stale-but-labeled file.
|
|
85
|
+
- **Promotion to static**: once a dynamic skill has proven itself general β used successfully across genuinely unrelated tasks, not repeated instances of the same bug β promote it: move it to the repo root, drop the `metadata.type: dynamic` block, and register it in `harness-everything/SKILL.md` Β§5 like any other skill. At that point it's subject to the same review/PR discipline as everything else here.
|
|
86
|
+
|
|
87
|
+
## 5. Related skills
|
|
88
|
+
|
|
89
|
+
- `skill-style/SKILL.md` β the terse Skill Contract format spec this skill builds on. Read it if you just need the table shape, not the full workflow.
|
|
90
|
+
- `create-agent-launcher/SKILL.md` β used in Β§2 Step 3 to spawn with/without-skill test subagents.
|
|
91
|
+
- `self-evolve/SKILL.md` β the caller for Β§4; it loads this skill rather than duplicating the Dynamic Skill Generation Contract inline.
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# Quality Principles β condensed from `writing-great-skills` and Anthropic's `skill-creator`
|
|
2
|
+
|
|
3
|
+
Read this when `skill-creator/SKILL.md`'s Quality Checklist flags something and you need the reasoning behind the check, not just the checkbox. Each term below is condensed from the two source skills and paired with a Harness-specific example β good or bad β found during the 2026-07-22 audit.
|
|
4
|
+
|
|
5
|
+
## Contents
|
|
6
|
+
- [The root idea: Predictability](#the-root-idea-predictability)
|
|
7
|
+
- [Information hierarchy](#information-hierarchy)
|
|
8
|
+
- [Pruning](#pruning)
|
|
9
|
+
- [Leading words](#leading-words)
|
|
10
|
+
- [Negation](#negation)
|
|
11
|
+
- [Invocation, adapted for Harness's router](#invocation-adapted-for-harnesss-router)
|
|
12
|
+
- [Anthropic `skill-creator`'s anatomy conventions](#anthropic-skill-creators-anatomy-conventions)
|
|
13
|
+
|
|
14
|
+
## The root idea: Predictability
|
|
15
|
+
|
|
16
|
+
A skill exists to make the agent take the same *process* on every run β not the same output. A skill that should diverge (brainstorming, `grill-me`'s interrogation) is still predictable if it reliably takes the same *kind* of path every time. Every other term here is a lever on this one thing. Harness reaches for the same goal through a different lever β a physical Enforcement Gate instead of a cognitive one β see `skill-creator/SKILL.md` Β§1 for how the two combine.
|
|
17
|
+
|
|
18
|
+
## Information hierarchy
|
|
19
|
+
|
|
20
|
+
Content in a skill ranks by how immediately the agent needs it:
|
|
21
|
+
|
|
22
|
+
1. **Steps** β ordered actions in the SKILL.md body. Each ends on a completion criterion. Harness's Skill Contract table makes this criterion unusually concrete by default (an exit code, a file write) β keep that when you write new steps; don't downgrade a checkable criterion to a vague one for the sake of shorter prose.
|
|
23
|
+
2. **In-file reference** β rules, tables, decision lists consulted on demand. A skill can be *all* reference with no steps at all (`skill-style` itself, most review-shaped skills) β that's a legitimate shape, not a smell.
|
|
24
|
+
3. **Disclosed / external reference** β pushed into a linked file, reached by a pointer, loaded only when the pointer fires.
|
|
25
|
+
|
|
26
|
+
**Good Harness example**: `git-commit/SKILL.md` keeps the core Angular-style rules inline and pushes submodule handling, language detection, and generation strategy into `git-commit/guides/*.md`, each named for what it holds. `tdd/SKILL.md` does the same with `tdd/guides/mocking.md`, `interface-design.md`, etc.
|
|
27
|
+
|
|
28
|
+
**Bad Harness example**: `security-review/SKILL.md` is 508 lines β the longest in the repo β with no reference-file split. It wasn't re-read line-by-line for this audit, but its length alone is the sprawl smell described below; it's the next candidate if this skill's checklist gets applied broadly.
|
|
29
|
+
|
|
30
|
+
A **context pointer**'s *wording* decides whether the agent actually opens the linked file β not just its existence. "See `guides/mocking.md` for more" is weak; "before writing any test double, read `guides/mocking.md`" is a pointer that actually fires. Every "Deep Reference Guides" section in this repo should name *when*, not just *what*.
|
|
31
|
+
|
|
32
|
+
## Pruning
|
|
33
|
+
|
|
34
|
+
Four related failure modes, roughly in order of how much they cost:
|
|
35
|
+
|
|
36
|
+
- **Duplication** β the same meaning stated in more than one place. Costs tokens on every load, and drifts over time because nobody remembers to edit both copies. The audit report's Β§1.1 (33 lines shared verbatim between `install-cognitive-os` and `harness-everything`) and Β§1.2 (one skill's description independently rewritten in four places, two of which already disagree) are the two worst confirmed cases in this repo. **Test**: could you point to exactly one place that defines this, and would editing only that place actually change the behavior everywhere it matters?
|
|
37
|
+
- **Sediment** β stale content nobody removes because adding feels safe and removing feels risky. `docs/workflows/skill-style.md`'s three Mermaid diagrams (audit Β§1.4) are a mild case: not wrong, just three restatements of one flow that accreted instead of being consolidated.
|
|
38
|
+
- **Sprawl** β a skill simply too long, even if every line is live and unique. The cure is always the information hierarchy above: push reference behind a pointer, split by branch.
|
|
39
|
+
- **No-op** β an instruction the model already follows by default, so you pay tokens to say nothing. Test it against the actual default, not against how important the rule feels: "MUST operate using the cognitive loop" repeated inside a sub-skill when `install-cognitive-os` already declares itself always-loaded (audit Β§2.3) is a borderline example β not egregious, but a candidate to cut the next time that file is touched.
|
|
40
|
+
|
|
41
|
+
## Leading words
|
|
42
|
+
|
|
43
|
+
A leading word is a compact, already-understood concept the model can think *with*, repeated as a token (not re-explained as a sentence) everywhere it applies. This repo already has strong ones β don't invent synonyms for them:
|
|
44
|
+
|
|
45
|
+
| Leading word | What it compresses |
|
|
46
|
+
|---|---|
|
|
47
|
+
| `Tier 1 / 2 / 3` | Task-complexity routing and the whole action-strategy that follows from each |
|
|
48
|
+
| `circuit breaker` / `zoom out` | The entire reflect-before-retry protocol in `zoom-out/SKILL.md` |
|
|
49
|
+
| `Rule of 3` | "three failed attempts on the same signature β stop and reflect" |
|
|
50
|
+
| `Red / Green / Refactor` | The TDD cycle and which actions are legal in each phase |
|
|
51
|
+
| `Discover > Think > Try > Summarize > Record` | The base cognitive loop every tier and skill runs inside |
|
|
52
|
+
|
|
53
|
+
When a new skill needs to say "stop and reflect on repeated failure," write `trigger zoom-out`, not a fresh paragraph re-explaining the reflection protocol. That's the whole payoff of a leading word: one anchored phrase instead of a restated idea.
|
|
54
|
+
|
|
55
|
+
## Negation
|
|
56
|
+
|
|
57
|
+
A prohibition drags the forbidden behavior into the model's attention rather than out of it β "don't think of an elephant" makes the elephant the only thing present. This repo uses `MUST NOT`/`PROHIBITED` heavily and by design (`skill-style` mandates it), so the actual lever available isn't "stop using negation" β it's **reserve it for gates that are real**, and pair the rest with the positive target:
|
|
58
|
+
|
|
59
|
+
- **Does this well**: `zoom-out/SKILL.md`'s "ABSOLUTELY PROHIBITED from proposing any new code modification" is immediately followed by *why* ("that sentence is the tunnel talking") and by what to do instead (Phase 2's read-only fact-checking). The prohibition isn't left to stand alone.
|
|
60
|
+
- **Could do better**: `environment-detection/SKILL.md`'s "Red Flags & Common Errors" section restates, as a list of β prohibitions, rules the same file already gave positively one section earlier (which shell uses which syntax). The information isn't wrong, it's just said twice in two moods.
|
|
61
|
+
|
|
62
|
+
If you're about to write `MUST NOT X`, check: is there a script or state mutation that actually enforces this, or human cost if violated (data loss, a broken build, a security gap)? If yes, keep the imperative β that's this repo's real strength, don't soften it. If no, try stating what *good* looks like instead and see if the `MUST NOT` was even carrying information.
|
|
63
|
+
|
|
64
|
+
## Invocation, adapted for Harness's router
|
|
65
|
+
|
|
66
|
+
The original framework's "model-invoked vs. user-invoked" split assumes a native skill system where an agent autonomously reads `description` fields and decides what to fire. That's not how this repo's skills are actually reached β confirmed by reading `scripts/installer.js` and `harness-everything/scripts/tier-router.js` directly rather than assuming:
|
|
67
|
+
|
|
68
|
+
- `harness-everything/SKILL.md` Β§5's Skill Registry table is a hand-maintained index β functionally Harness's own version of the framework's **router skill** pattern (a single place a human, or `tier-router.js` on their behalf, can look up "what exists and when to reach for it").
|
|
69
|
+
- `tier-router.js` prints keyword-triggered recommendations from its own hardcoded strings β a second, independent "what does this skill do" list that must be kept in sync with the registry and the frontmatter by hand (this is where the drift in audit Β§1.2 came from).
|
|
70
|
+
- `scripts/installer.js`'s interactive CLI picker is the one place that *does* read frontmatter `description` directly and shows it to a live reader β a **human**, choosing checkboxes at install time. That makes the real invocation axis in this repo closer to "does a human need to understand this in one line while skimming a checklist" than "will an autonomous model decide to fire this."
|
|
71
|
+
|
|
72
|
+
Practical upshot: write `description` as a plain, scannable sentence for that installer screen. Trigger-condition phrasing ("Use when...") isn't wrong, but it's optimizing for a reader (an autonomous picker) that doesn't exist in this deployment yet. If this repo's skills are ever installed as native Claude Skills (`.claude/skills/`) rather than the current `.claude/harness-everything/skills` / hook-advisory model, this section should be revisited β the trigger-phrasing advice becomes directly relevant again at that point.
|
|
73
|
+
|
|
74
|
+
## Anthropic `skill-creator`'s anatomy conventions
|
|
75
|
+
|
|
76
|
+
Kept because they don't conflict with anything above β they're just concrete, useful defaults:
|
|
77
|
+
|
|
78
|
+
- Bundled resources split by purpose: `scripts/` for things that execute deterministically, `references/` for things loaded into context on demand (this file is one), `guides/`/`templates/` (Harness's existing naming) for the same idea.
|
|
79
|
+
- A reference file past ~300 lines gets a table of contents (this file has one).
|
|
80
|
+
- Domain-organized skills (a skill that supports several distinct variants) split by variant, one file per variant, so only the relevant one loads β `repo-docs/templates/*.md` already does this for different README/AGENTS.md archetypes.
|
|
81
|
+
- Draft first, then re-read with fresh eyes and cut. Don't try to write the pruned version on the first pass.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Testing Workflow β adapted from Anthropic's `skill-creator` eval loop
|
|
2
|
+
|
|
3
|
+
Read this when `skill-creator/SKILL.md` Β§2 Step 3 says "test against real prompts" and you need the mechanics. Anthropic's `skill-creator` runs this through a Python eval-viewer with subagent grading and a `claude -p`-based description-optimization loop. Harness doesn't import that toolchain β it already has `create-agent-launcher` for resource-isolated subagent spawning, and its skill-routing is deterministic keyword matching rather than autonomous LLM judgment, so the equivalent checks are cheaper and don't need a second dependency.
|
|
4
|
+
|
|
5
|
+
## When to bother
|
|
6
|
+
|
|
7
|
+
Not every skill needs this. A short, low-stakes reference skill (a rule table, a style guide) is fine to ship on a careful re-read. Run the full loop when:
|
|
8
|
+
- The skill has real **Steps** the agent must execute in a specific order, especially ones with an Enforcement Gate β you want to see the gate actually fire on a bad run, not just assume it will.
|
|
9
|
+
- The skill is going to be recommended automatically (registered in `harness-everything/SKILL.md` Β§5 and/or `tier-router.js`) rather than only invoked by name β automatic recommendation means it'll be loaded in situations you didn't personally test by hand.
|
|
10
|
+
- You're refactoring an existing skill based on a specific complaint, and want to confirm the fix actually lands rather than just looking right on the page.
|
|
11
|
+
|
|
12
|
+
## With/without comparison via `create-agent-launcher`
|
|
13
|
+
|
|
14
|
+
1. Write 2-3 prompts a real user would actually type β concrete, with the kind of incidental detail real requests carry (a file path, a framework name, a half-finished thought). Anthropic's `skill-creator` has good guidance on this: avoid both sides being trivially obvious. A bad "should trigger" prompt is abstract ("test the skill"); a bad "should not trigger" prompt is unrelated to anything the skill touches. The useful test cases are near-misses in both directions.
|
|
15
|
+
2. Per `create-agent-launcher/SKILL.md` Β§2 (Persona Definition, Resource Isolation): spawn one subagent with the draft skill's file path given to it explicitly and instructed to follow it, and one baseline subagent given the same prompt with no mention of the skill.
|
|
16
|
+
3. Compare outputs side by side, but **read the transcripts, not just the final result** β a skill that produces a fine final answer while making the agent thrash through three dead-end tool calls first is still under-specifying something; the final output alone won't show you that.
|
|
17
|
+
4. Where the skill has a real Enforcement Gate, deliberately construct one test case that should trip it (e.g., for a skill gating on a failing test, hand it a prompt implying the test currently fails) and confirm the with-skill run actually stops/reflects instead of pushing through.
|
|
18
|
+
|
|
19
|
+
## Checking trigger phrasing against a deterministic router
|
|
20
|
+
|
|
21
|
+
Anthropic's `skill-creator` has a whole optimization loop (`run_loop.py`) that runs a description against dozens of realistic queries through the actual model, because native Claude Skills decide to fire based on an LLM reading the description at run time. Harness's routing recommendation layer (`tier-router.js`) is **not** an LLM judgment call β it's `promptLower.includes(keyword)` string matching against hardcoded arrays. That means you can check trigger coverage deterministically, without spawning a single subagent:
|
|
22
|
+
|
|
23
|
+
1. Write the same kind of realistic should-trigger / should-not-trigger prompt set described above (8-10 each is plenty for a new skill; fewer is fine for a small change).
|
|
24
|
+
2. For each should-trigger prompt, check by eye (or a one-line `node -e` snippet) whether it contains at least one keyword from the relevant array in `harness-everything/scripts/tier-router.js`. If a realistic phrasing doesn't hit any keyword, that's a real gap β add the missing term (bilingual, if this repo's convention applies β most keyword arrays here carry both English and Chinese terms side by side).
|
|
25
|
+
3. For each should-not-trigger prompt β especially near-misses that share vocabulary with the skill but need something else β confirm it does *not* spuriously match. A keyword that's too broad (a single common English word with no qualifying context) is the usual cause of a false positive; narrow it or require it alongside a second signal.
|
|
26
|
+
4. This check is cheap enough to re-run by hand every time a keyword array changes β there's no reason to skip it even for a one-line addition.
|
|
27
|
+
|
|
28
|
+
This isn't a replacement for the with/without subagent comparison above β it only tells you whether the skill gets *recommended* at the right moments, not whether it's good once loaded. Both checks matter; they're checking different failure modes.
|
|
29
|
+
|
|
30
|
+
## Folding feedback back in
|
|
31
|
+
|
|
32
|
+
If a human reviewer (or your own re-read) flags something in the output, resist the urge to patch the skill with a rule that only fixes that one test case. Ask what the *general* version of the complaint is first β the same discipline `skill-creator/SKILL.md` Β§3's Quality Checklist asks for: is this a genuinely missing rule, or does an existing rule already cover it and just needs a stronger leading word to actually land? Re-test after any change, including the baseline β a skill's environment (the rest of the repo, the router keywords around it) can shift under it between iterations.
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: skill-style
|
|
3
|
+
description: Guidelines for writing and refactoring skills in the Harness ecosystem.
|
|
4
|
+
author: Miya Daniel | Harness Core Team
|
|
5
|
+
version: 0.2.0
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Skill Style (Harness Ecosystem Skill Development Guidelines)
|
|
9
|
+
|
|
10
|
+
When you need to create a new Skill or refactor an existing one, strictly adhere to the writing style in this document to ensure all Skills seamlessly integrate into the `harness-everything` routing system and are mathematically enforced by scripts.
|
|
11
|
+
|
|
12
|
+
## 0. Triggers
|
|
13
|
+
This skill should be loaded when:
|
|
14
|
+
- Creating a new SKILL.md file.
|
|
15
|
+
- Refactoring or reviewing an existing SKILL.md file.
|
|
16
|
+
|
|
17
|
+
## 1. Structured Definition
|
|
18
|
+
Every Skill file (`SKILL.md`) MUST contain the following standard structure:
|
|
19
|
+
|
|
20
|
+
1. **Title & Introduction**: Clearly state the purpose of this Skill.
|
|
21
|
+
2. **π Skill Contract (CRITICAL)**: A strict Markdown table defining inputs, outputs, state mutations, and script gates. This replaces vague prose.
|
|
22
|
+
3. **Triggers / Usage**: Explicitly tell the system when this skill applies.
|
|
23
|
+
4. **Core Rules / Action List**: Specific, actionable commands tied to Terminal Scripts. Avoid vague suggestions.
|
|
24
|
+
|
|
25
|
+
### π The Skill Contract Format
|
|
26
|
+
Every SKILL.md MUST include this table exactly:
|
|
27
|
+
|
|
28
|
+
```markdown
|
|
29
|
+
## π Skill Contract
|
|
30
|
+
|
|
31
|
+
| Component | Specification |
|
|
32
|
+
| :--- | :--- |
|
|
33
|
+
| **Trigger / Input** | (What causes this skill to execute? What exact data/args does it expect?) |
|
|
34
|
+
| **Expected Output** | (What specific files, artifacts, or Terminal Exit Codes are produced?) |
|
|
35
|
+
| **State Mutations** | (What JSON/files are written to track progress? e.g., `.claude/harness-state/todo-state.json`) |
|
|
36
|
+
| **Enforcement Gate** | (Which exact CLI script slaps the LLM with Exit Code 1 if it fails?) |
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
## 2. Tone & Voice
|
|
40
|
+
- **Anti-Linear / Anti-Prose**: Do not write long paragraphs. Map actions to specific `run_in_terminal` commands.
|
|
41
|
+
- **Absolute Imperatives**: Use "MUST", "MUST NOT", "ALWAYS". Do not use "suggest".
|
|
42
|
+
- **Script-Driven Enforcement**: Do not write "You should check your code". Write "You MUST run `node verify-gate.js`. If Exit Code 1, you MUST reflect and retry."
|
|
43
|
+
|
|
44
|
+
## 3. Avoid Functional Overlap
|
|
45
|
+
- **OS Skills vs. Domain Skills**: Distinguish between the OS layer (which routes and constrains behavior) and the Domain layer (which provides deep technical expertise).
|
|
46
|
+
|
|
47
|
+
## 4. For the fuller authoring & quality workflow, see `skill-creator`
|
|
48
|
+
This document is the terse format spec β the Skill Contract table shape and the tone rules. For interviewing intent, drafting, testing a skill against real prompts, pruning duplication/no-op/sprawl, and the rules for skills `self-evolve` generates dynamically mid-session, load `skill-creator/SKILL.md` instead. It builds on this spec rather than replacing it.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# SKILL Document Writing Guidelines
|
|
2
|
+
|
|
3
|
+
> Target: Markdown files read and executed as Prompts during SKILL execution (excludes Template files).
|
|
4
|
+
|
|
5
|
+
## [Language & Syntax]
|
|
6
|
+
1. All content MUST be in English (All English).
|
|
7
|
+
2. MUST exclusively use task-based syntax (Imperative verbs, numbered lists).
|
|
8
|
+
3. Vague tone is strictly forbidden. MUST use uppercase `MUST` or `MUST NOT` to define behaviors. Words like `should` or `may` are forbidden.
|
|
9
|
+
4. The length of each rule MUST NOT exceed 150 characters.
|
|
10
|
+
|
|
11
|
+
## [Skill Architecture: OS vs Domain]
|
|
12
|
+
5. **OS Skills (Process Control)**: e.g., `install-cognitive-os`, `harness-everything`. These dictate *how* to work. They MUST be strictly linear and MUST NOT exceed 60 lines per file.
|
|
13
|
+
6. **Domain Skills (Expertise)**: e.g., `api-design`, `security-review`. These provide *what* to write. They are EXEMPT from the 60-line and linearity limits. They are encouraged to be comprehensive, containing rich anti-patterns, code examples, and deep domain guardrails.
|
|
14
|
+
|
|
15
|
+
## [Flow & Linearity (Applies to OS Skills)]
|
|
16
|
+
7. The SKILL entry point (SKILL.md) MUST be executed only once.
|
|
17
|
+
8. The flow from OS SKILL.md to the end MUST be a flattened straight line. Infinite loops or branching trees are forbidden.
|
|
18
|
+
9. All OS files MUST form a logical closed loop. If split, the current file MUST end with a unique pointer to maintain linearity.
|
|
19
|
+
|
|
20
|
+
## [Cognitive Loop & Execution]
|
|
21
|
+
9. Steps MUST be written sequentially. Every SKILL file MUST explicitly include this declaration:
|
|
22
|
+
```markdown
|
|
23
|
+
## Core Principles
|
|
24
|
+
- MUST operate using the cognitive loop: Think > Try > Summarize > Record.
|
|
25
|
+
- [Think] MUST build a holistic mental router map before starting. Use elimination to discard impossible approaches. Perform forward prediction to simplify actions.
|
|
26
|
+
- [Try] MUST verify current environment boundaries BEFORE taking action. Execute steps sequentially and NEVER combine commands, ensuring errors remain isolated and traceable.
|
|
27
|
+
- [Summarize] MUST verify actual outcomes against initial intent; hallucinating success is forbidden. If an attempt fails, diagnose, zoom out, and backtrack.
|
|
28
|
+
- [Record] MUST explicitly state variables and context to carry over.
|
|
29
|
+
- [Reflect & Re-Think] MUST actively feed the `[Record]` (especially past failures and state changes) back into the next `[Think]` cycle. Use historical context to refine forward predictions and strictly avoid repeating previously diagnosed errors.
|
|
30
|
+
|
|
31
|
+
## Cognitive Foundation (Why we do this)
|
|
32
|
+
- Recording is not the end; it is the absolute prerequisite for the next Think cycle.
|
|
33
|
+
- Trying validates the predicted steps.
|
|
34
|
+
- Summarizing confirms if the outcome aligns with the initial intent.
|
|
35
|
+
- The loop is continuous. Converging the chain of thought is the absolute priority; divergent thinking causes attention loss. This cycle (Think > Try > Summarize > Record > Re-Think) drives self-adaptation and evolution.
|
|
36
|
+
- Treat these rules as strict guardrails. Autonomous optimization within these boundaries is expected.
|
|
37
|
+
```
|
package/tdd/SKILL.md
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: tdd
|
|
3
|
+
description: Test-Driven Development mode (Red-Green-Refactor) for Standard Tasks (Tier 2).
|
|
4
|
+
author: Miya Daniel | Harness Core Team
|
|
5
|
+
version: 0.2.0
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Test-Driven Development (TDD) Mode
|
|
9
|
+
|
|
10
|
+
## π Skill Contract
|
|
11
|
+
|
|
12
|
+
| Component | Specification |
|
|
13
|
+
| :--- | :--- |
|
|
14
|
+
| **Trigger / Input** | Tier 2 Task identification. Input: The specific feature or bug requirement. |
|
|
15
|
+
| **Expected Output** | 1. Failing test terminal output. 2. Implementation code. 3. Passing test terminal output (Exit Code 0). |
|
|
16
|
+
| **State Mutations** | Handled by `todo-cli.js` (tracked via `.claude/harness-state/todo-state.json`). |
|
|
17
|
+
| **Enforcement Gate** | You MUST run the test runner in the terminal. If it does not fail first (RED), you MUST fix the test. If it fails later (GREEN phase), you MUST reflect and fix the code. |
|
|
18
|
+
|
|
19
|
+
This skill is automatically triggered and loaded by the `harness-everything` router when a task is judged as a **Tier 2 (Standard Task)**.
|
|
20
|
+
It applies to adding a single feature, fixing a specific Bug, or medium-sized changes with clear expected outcomes.
|
|
21
|
+
|
|
22
|
+
Tier 2 tasks run on the `todo-driven-workflow` base execution loop: initialize the checklist first, and map each Red / Green / Refactor phase (per feature or bug) onto its own verifiable todo item.
|
|
23
|
+
|
|
24
|
+
## TDD Core Discipline (Red-Green-Refactor)
|
|
25
|
+
|
|
26
|
+
When this skill is loaded, you MUST suppress the urge to write implementation code directly, and strictly follow these three phases:
|
|
27
|
+
|
|
28
|
+
### 1. RED (Write a Failing Test)
|
|
29
|
+
- **Action**: Before implementing the requested feature or Bug fix, write the corresponding Unit Test or Integration Test in the test folder.
|
|
30
|
+
- **Validation**: Run the test to **ensure the test fails** (this proves the test actually covers unimplemented functionality, rather than being a fake test).
|
|
31
|
+
- **Note**: If the test passes immediately, it means your test is wrong or the Bug doesn't actually exist. You must fix the test.
|
|
32
|
+
|
|
33
|
+
### 2. GREEN (Implement Minimal Code to Pass Test)
|
|
34
|
+
- **Action**: Switch to the implementation code and write the "minimum amount of code just enough to pass the test". Do not over-engineer or consider future extensibility at this stage.
|
|
35
|
+
- **Validation**: Run the test to ensure it passes.
|
|
36
|
+
|
|
37
|
+
### 3. REFACTOR (Refactor and Optimize)
|
|
38
|
+
- **Action**: Under the safety net of passing tests, begin optimizing the code.
|
|
39
|
+
- Checks: Is the naming clear? Is there duplicated code? Can performance be improved? Does it comply with the project's Clean Code standards?
|
|
40
|
+
- **Validation**: After every modification, re-run the tests to ensure refactoring hasn't broken the original functionality.
|
|
41
|
+
|
|
42
|
+
## Circuit Breaker Defense
|
|
43
|
+
If during a TDD cycle you get stuck in the **GREEN phase**, and 3 consecutive implementation attempts fail to pass the test:
|
|
44
|
+
- **Trigger Condition Met**: You might have hit the reasoning ceiling, or the initial test logic (RED) was written incorrectly.
|
|
45
|
+
- **Mandatory Action**: Stop guessing blindly. Immediately abort the TDD process and call the `zoom-out` skill: fact-check the assumption behind each failing attempt β including whether the RED test itself encodes the wrong expectation β and resume on a fresh diagnosis. Bring the human in only if the reflection surfaces a genuine decision (e.g., the test contradicts the requirement).
|
|
46
|
+
|
|
47
|
+
## Deep Reference Guides
|
|
48
|
+
For advanced testing, design, and mocking strategies, you MUST refer to:
|
|
49
|
+
- `tdd/guides/mocking.md` β Mocking and stubbing principles
|
|
50
|
+
- `tdd/guides/interface-design.md` β Interface and contract-driven design
|
|
51
|
+
- `tdd/guides/deep-modules.md` β Testing deeply nested/complex modules
|
|
52
|
+
- `tdd/guides/tests.md` β General test architecture and assertions
|
|
53
|
+
- `tdd/guides/refactoring.md` β Refactoring techniques under safety nets
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Deep Modules
|
|
2
|
+
|
|
3
|
+
From "A Philosophy of Software Design":
|
|
4
|
+
|
|
5
|
+
**Deep module** = small interface + lots of implementation
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
βββββββββββββββββββββββ
|
|
9
|
+
β Small Interface β β Few methods, simple params
|
|
10
|
+
βββββββββββββββββββββββ€
|
|
11
|
+
β β
|
|
12
|
+
β β
|
|
13
|
+
β Deep Implementationβ β Complex logic hidden
|
|
14
|
+
β β
|
|
15
|
+
β β
|
|
16
|
+
βββββββββββββββββββββββ
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
**Shallow module** = large interface + little implementation (avoid)
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
βββββββββββββββββββββββββββββββββββ
|
|
23
|
+
β Large Interface β β Many methods, complex params
|
|
24
|
+
βββββββββββββββββββββββββββββββββββ€
|
|
25
|
+
β Thin Implementation β β Just passes through
|
|
26
|
+
βββββββββββββββββββββββββββββββββββ
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
When designing interfaces, ask:
|
|
30
|
+
|
|
31
|
+
- Can I reduce the number of methods?
|
|
32
|
+
- Can I simplify the parameters?
|
|
33
|
+
- Can I hide more complexity inside?
|