@cleocode/cleo 2026.3.2 → 2026.3.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -3
- package/dist/cli/index.js +7381 -3507
- package/dist/cli/index.js.map +4 -4
- package/dist/mcp/index.js +5792 -2097
- package/dist/mcp/index.js.map +4 -4
- package/package.json +4 -4
- package/packages/ct-skills/index.js +1 -1
- package/packages/ct-skills/package.json +0 -2
- package/packages/ct-skills/profiles/core.json +1 -1
- package/packages/ct-skills/profiles/full.json +4 -5
- package/packages/ct-skills/profiles/minimal.json +3 -3
- package/packages/ct-skills/profiles/recommended.json +2 -2
- package/packages/ct-skills/provider-skills-map.json +97 -0
- package/packages/ct-skills/skills/_shared/skill-chaining-patterns.md +23 -26
- package/packages/ct-skills/skills/_shared/testing-framework-config.md +9 -9
- package/packages/ct-skills/skills/ct-cleo/SKILL.md +21 -1
- package/packages/ct-skills/skills/ct-dev-workflow/SKILL.md +1 -1
- package/packages/ct-skills/skills/ct-documentor/SKILL.md +1 -1
- package/packages/ct-skills/skills/ct-epic-architect/SKILL.md +1 -1
- package/packages/ct-skills/skills/ct-grade/SKILL.md +214 -0
- package/packages/ct-skills/skills/ct-orchestrator/SKILL.md +119 -43
- package/packages/ct-skills/skills/ct-orchestrator/orchestrator-prompt.txt +17 -0
- package/packages/ct-skills/skills/ct-orchestrator/references/orchestrator-patterns.md +1 -1
- package/packages/ct-skills/skills/ct-research-agent/SKILL.md +1 -1
- package/packages/ct-skills/skills/ct-spec-writer/SKILL.md +1 -1
- package/packages/ct-skills/skills/ct-task-executor/SKILL.md +1 -1
- package/packages/ct-skills/skills/ct-validator/SKILL.md +1 -1
- package/packages/ct-skills/skills/manifest.json +217 -947
- package/packages/ct-skills/skills.json +244 -3
- package/templates/CLEO-INJECTION.md +37 -0
- package/templates/README.md +4 -4
- package/templates/cleo-gitignore +3 -11
- package/templates/git-hooks/pre-commit +1 -1
- package/packages/ct-skills/protocols/agent-protocol.md +0 -260
- package/packages/ct-skills/protocols/artifact-publish.md +0 -587
- package/packages/ct-skills/protocols/consensus.md +0 -309
- package/packages/ct-skills/protocols/contribution.md +0 -375
- package/packages/ct-skills/protocols/decomposition.md +0 -352
- package/packages/ct-skills/protocols/implementation.md +0 -344
- package/packages/ct-skills/protocols/provenance.md +0 -600
- package/packages/ct-skills/protocols/release.md +0 -635
- package/packages/ct-skills/protocols/research.md +0 -248
- package/packages/ct-skills/protocols/specification.md +0 -287
- package/packages/ct-skills/protocols/testing.md +0 -346
- package/packages/ct-skills/protocols/validation.md +0 -229
- package/packages/ct-skills/skills/ct-gitbook/SKILL.md +0 -516
- package/packages/ct-skills/skills/ct-gitbook/assets/SUMMARY.md +0 -28
- package/packages/ct-skills/skills/ct-gitbook/assets/gitbook.yaml +0 -14
- package/packages/ct-skills/skills/ct-gitbook/references/api-sdk.md +0 -318
- package/packages/ct-skills/skills/ct-gitbook/references/auth-sso.md +0 -208
- package/packages/ct-skills/skills/ct-gitbook/references/change-requests.md +0 -169
- package/packages/ct-skills/skills/ct-gitbook/references/content-blocks.md +0 -230
- package/packages/ct-skills/skills/ct-gitbook/references/docs-sites.md +0 -202
- package/packages/ct-skills/skills/ct-gitbook/references/git-sync.md +0 -175
- package/packages/ct-skills/skills/ct-gitbook/references/llm-ready.md +0 -178
- package/packages/ct-skills/skills/ct-gitbook/references/migration.md +0 -263
- package/packages/ct-skills/skills/ct-library-implementer-bash/SKILL.md +0 -316
- package/packages/ct-skills/skills/ct-skill-lookup/SKILL.md +0 -179
- package/packages/ct-skills/skills/ct-test-writer-bats/SKILL.md +0 -347
- package/packages/ct-skills/skills/railway-platform/SKILL.md +0 -506
- package/packages/ct-skills/skills/railway-platform/_shared/scripts/railway-api.sh +0 -180
- package/packages/ct-skills/skills/railway-platform/_shared/scripts/railway-common.sh +0 -262
- package/packages/ct-skills/skills/railway-platform/references/01-getting-started.md +0 -149
- package/packages/ct-skills/skills/railway-platform/references/02-projects.md +0 -116
- package/packages/ct-skills/skills/railway-platform/references/03-services.md +0 -147
- package/packages/ct-skills/skills/railway-platform/references/04-deployments.md +0 -210
- package/packages/ct-skills/skills/railway-platform/references/05-databases.md +0 -142
- package/packages/ct-skills/skills/railway-platform/references/06-environments.md +0 -261
- package/packages/ct-skills/skills/railway-platform/references/07-domains.md +0 -139
- package/packages/ct-skills/skills/railway-platform/references/08-volumes.md +0 -533
- package/packages/ct-skills/skills/railway-platform/references/09-networking.md +0 -592
- package/packages/ct-skills/skills/railway-platform/references/10-cron.md +0 -488
- package/packages/ct-skills/skills/railway-platform/references/11-functions.md +0 -170
- package/packages/ct-skills/skills/railway-platform/references/12-monorepo.md +0 -294
- package/packages/ct-skills/skills/railway-platform/references/13-troubleshooting.md +0 -335
- package/packages/ct-skills/skills/railway-platform/references/14-railway-metal.md +0 -197
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cleocode/cleo",
|
|
3
|
-
"version": "2026.3.
|
|
3
|
+
"version": "2026.3.6",
|
|
4
4
|
"description": "CLEO V2 - TypeScript task management CLI for AI coding agents",
|
|
5
5
|
"mcpName": "io.github.kryptobaseddev/cleo-mcp-server",
|
|
6
6
|
"type": "module",
|
|
@@ -24,9 +24,9 @@
|
|
|
24
24
|
"dev": "tsc --noEmit --watch",
|
|
25
25
|
"dev:setup": "bash dev/setup-ts-dev.sh",
|
|
26
26
|
"dev:watch": "node build.mjs --watch",
|
|
27
|
-
"test": "vitest run",
|
|
28
|
-
"test:watch": "vitest",
|
|
29
|
-
"test:coverage": "vitest run --coverage",
|
|
27
|
+
"test": "LOG_LEVEL=silent NODE_NO_WARNINGS=1 vitest run",
|
|
28
|
+
"test:watch": "LOG_LEVEL=silent NODE_NO_WARNINGS=1 vitest",
|
|
29
|
+
"test:coverage": "LOG_LEVEL=silent NODE_NO_WARNINGS=1 vitest run --coverage",
|
|
30
30
|
"lint": "tsc --noEmit",
|
|
31
31
|
"prepare": "npm run build",
|
|
32
32
|
"prepack": "npm run build",
|
|
@@ -6,7 +6,7 @@ const fs = require('fs');
|
|
|
6
6
|
const LIBRARY_ROOT = __dirname;
|
|
7
7
|
const SKILLS_ROOT = path.join(LIBRARY_ROOT, 'skills');
|
|
8
8
|
const PROFILES_ROOT = path.join(LIBRARY_ROOT, 'profiles');
|
|
9
|
-
const PROTOCOLS_ROOT = path.join(LIBRARY_ROOT, 'protocols');
|
|
9
|
+
const PROTOCOLS_ROOT = path.join(LIBRARY_ROOT, '..', '..', 'src', 'protocols');
|
|
10
10
|
const SHARED_ROOT = path.join(SKILLS_ROOT, '_shared');
|
|
11
11
|
|
|
12
12
|
// --- Package metadata ---
|
|
@@ -1,11 +1,10 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "full",
|
|
3
|
-
"description": "All
|
|
3
|
+
"description": "All skills including execution, documentation, quality, and meta",
|
|
4
4
|
"extends": "recommended",
|
|
5
5
|
"skills": [
|
|
6
|
-
"ct-dev-workflow", "ct-
|
|
7
|
-
"ct-
|
|
8
|
-
"ct-skill-creator", "ct-skill-lookup", "ct-contribution", "ct-gitbook"
|
|
6
|
+
"ct-dev-workflow", "ct-documentor", "ct-docs-lookup", "ct-docs-write", "ct-docs-review",
|
|
7
|
+
"ct-contribution", "ct-grade", "ct-skill-creator"
|
|
9
8
|
],
|
|
10
|
-
"includeProtocols": ["contribution", "release", "testing", "artifact-publish", "provenance"]
|
|
9
|
+
"includeProtocols": ["implementation", "contribution", "release", "testing", "artifact-publish", "provenance"]
|
|
11
10
|
}
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "minimal",
|
|
3
|
-
"description": "
|
|
4
|
-
"skills": ["ct-task-executor"],
|
|
3
|
+
"description": "Solo task execution with core CLEO protocol — no orchestration needed",
|
|
4
|
+
"skills": ["ct-cleo", "ct-task-executor"],
|
|
5
5
|
"includeShared": true,
|
|
6
|
-
"includeProtocols": [
|
|
6
|
+
"includeProtocols": []
|
|
7
7
|
}
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "recommended",
|
|
3
|
-
"description": "
|
|
3
|
+
"description": "Full RCASD pipeline skills for epic-driven development",
|
|
4
4
|
"extends": "core",
|
|
5
5
|
"skills": ["ct-epic-architect", "ct-research-agent", "ct-spec-writer", "ct-validator"],
|
|
6
|
-
"includeProtocols": ["research", "specification", "decomposition", "
|
|
6
|
+
"includeProtocols": ["research", "consensus", "specification", "decomposition", "validation", "adr"]
|
|
7
7
|
}
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://lafs.dev/schemas/v1/provider-skills-map.schema.json",
|
|
3
|
+
"version": "1.0.0",
|
|
4
|
+
"description": "Provider skill path discovery and precedence rules for CLEO skill deployment",
|
|
5
|
+
"providers": {
|
|
6
|
+
"claude-code": {
|
|
7
|
+
"globalSkills": "$HOME/.claude/skills",
|
|
8
|
+
"projectSkills": ".claude/skills",
|
|
9
|
+
"agentsGlobal": null,
|
|
10
|
+
"agentsProject": null,
|
|
11
|
+
"readsFromDotAgents": false,
|
|
12
|
+
"precedence": "vendor-only",
|
|
13
|
+
"symlinkSupported": true,
|
|
14
|
+
"notes": "Created Agent Skills standard but uses vendor path exclusively"
|
|
15
|
+
},
|
|
16
|
+
"codex-cli": {
|
|
17
|
+
"globalSkills": "$HOME/.agents/skills",
|
|
18
|
+
"projectSkills": ".agents/skills",
|
|
19
|
+
"agentsGlobal": "$HOME/.agents/skills",
|
|
20
|
+
"agentsProject": ".agents/skills",
|
|
21
|
+
"readsFromDotAgents": true,
|
|
22
|
+
"precedence": "agents-canonical",
|
|
23
|
+
"symlinkSupported": true,
|
|
24
|
+
"notes": "Uses .agents/skills as canonical location"
|
|
25
|
+
},
|
|
26
|
+
"gemini-cli": {
|
|
27
|
+
"globalSkills": "$HOME/.gemini/skills",
|
|
28
|
+
"projectSkills": ".gemini/skills",
|
|
29
|
+
"agentsGlobal": "$HOME/.agents/skills",
|
|
30
|
+
"agentsProject": ".agents/skills",
|
|
31
|
+
"readsFromDotAgents": true,
|
|
32
|
+
"precedence": "agents-first",
|
|
33
|
+
"symlinkSupported": true,
|
|
34
|
+
"notes": ".agents/skills takes precedence over .gemini/skills within same tier"
|
|
35
|
+
},
|
|
36
|
+
"cursor": {
|
|
37
|
+
"globalSkills": "$HOME/.cursor/skills",
|
|
38
|
+
"projectSkills": ".cursor/skills",
|
|
39
|
+
"agentsGlobal": null,
|
|
40
|
+
"agentsProject": ".agents/skills",
|
|
41
|
+
"readsFromDotAgents": "project-only",
|
|
42
|
+
"precedence": "vendor-global-agents-project",
|
|
43
|
+
"symlinkSupported": true,
|
|
44
|
+
"notes": "Global uses vendor path; project supports .agents/skills"
|
|
45
|
+
},
|
|
46
|
+
"github-copilot": {
|
|
47
|
+
"globalSkills": "$HOME/.copilot/skills",
|
|
48
|
+
"projectSkills": ".github/skills",
|
|
49
|
+
"agentsGlobal": "$HOME/.agents/skills",
|
|
50
|
+
"agentsProject": ".agents/skills",
|
|
51
|
+
"readsFromDotAgents": true,
|
|
52
|
+
"precedence": "agents-supported",
|
|
53
|
+
"symlinkSupported": true,
|
|
54
|
+
"notes": "Supports both vendor and .agents paths at all levels"
|
|
55
|
+
},
|
|
56
|
+
"windsurf": {
|
|
57
|
+
"globalSkills": "$HOME/.codeium/windsurf/skills",
|
|
58
|
+
"projectSkills": ".windsurf/skills",
|
|
59
|
+
"agentsGlobal": null,
|
|
60
|
+
"agentsProject": null,
|
|
61
|
+
"readsFromDotAgents": false,
|
|
62
|
+
"precedence": "vendor-only",
|
|
63
|
+
"symlinkSupported": false,
|
|
64
|
+
"notes": "Vendor-specific paths only, no .agents support"
|
|
65
|
+
},
|
|
66
|
+
"opencode": {
|
|
67
|
+
"globalSkills": "$HOME/.config/opencode/skills",
|
|
68
|
+
"projectSkills": ".opencode/skills",
|
|
69
|
+
"agentsGlobal": "$HOME/.agents/skills",
|
|
70
|
+
"agentsProject": ".agents/skills",
|
|
71
|
+
"readsFromDotAgents": true,
|
|
72
|
+
"precedence": "agents-supported",
|
|
73
|
+
"symlinkSupported": true,
|
|
74
|
+
"notes": "User confirms .agents/ support; verify via opencode docs"
|
|
75
|
+
},
|
|
76
|
+
"kimi-coding": {
|
|
77
|
+
"globalSkills": "$HOME/.kimi/skills",
|
|
78
|
+
"projectSkills": ".kimi/skills",
|
|
79
|
+
"agentsGlobal": null,
|
|
80
|
+
"agentsProject": null,
|
|
81
|
+
"readsFromDotAgents": false,
|
|
82
|
+
"precedence": "vendor-only",
|
|
83
|
+
"symlinkSupported": false,
|
|
84
|
+
"notes": "Model-based provider; skill path support uncertain"
|
|
85
|
+
},
|
|
86
|
+
"antigravity": {
|
|
87
|
+
"globalSkills": "$HOME/.antigravity/skills",
|
|
88
|
+
"projectSkills": ".agent/skills",
|
|
89
|
+
"agentsGlobal": null,
|
|
90
|
+
"agentsProject": ".agents/skills",
|
|
91
|
+
"readsFromDotAgents": "project-only",
|
|
92
|
+
"precedence": "vendor-global-agents-project",
|
|
93
|
+
"symlinkSupported": true,
|
|
94
|
+
"notes": "Project-level .agents/skills confirmed in Google codelabs"
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
}
|
|
@@ -16,7 +16,7 @@ This reference defines patterns for multi-level skill invocation and context pro
|
|
|
16
16
|
|
|
17
17
|
## Pattern 1: Single-Level Spawning
|
|
18
18
|
|
|
19
|
-
The orchestrator delegates work to a subagent via
|
|
19
|
+
The orchestrator delegates work to a subagent via `orchestrate.spawn` with skill injection.
|
|
20
20
|
|
|
21
21
|
### Flow
|
|
22
22
|
|
|
@@ -25,7 +25,7 @@ The orchestrator delegates work to a subagent via Task tool with skill injection
|
|
|
25
25
|
│ ORCHESTRATOR │
|
|
26
26
|
│ (ct-orchestrator)
|
|
27
27
|
└────────┬────────┘
|
|
28
|
-
│
|
|
28
|
+
│ orchestrate.spawn → provider adapter
|
|
29
29
|
▼
|
|
30
30
|
┌─────────────────┐
|
|
31
31
|
│ SUBAGENT │
|
|
@@ -35,18 +35,15 @@ The orchestrator delegates work to a subagent via Task tool with skill injection
|
|
|
35
35
|
|
|
36
36
|
### Implementation
|
|
37
37
|
|
|
38
|
-
```
|
|
39
|
-
#
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
#
|
|
47
|
-
template=$(ti_load_template "skills/ct-research-agent/SKILL.md")
|
|
48
|
-
|
|
49
|
-
# Spawn via Task tool (includes subagent protocol block)
|
|
38
|
+
```
|
|
39
|
+
# 1. Generate fully-resolved spawn prompt via MCP
|
|
40
|
+
cleo_mutate({ domain: "orchestrate", operation: "spawn", params: { taskId: "T1234" }})
|
|
41
|
+
|
|
42
|
+
# 2. Provider adapter executes the prompt using its native mechanism
|
|
43
|
+
# - Claude Code: Task tool with cleo-subagent type
|
|
44
|
+
# - OpenCode: config-driven agent spawn
|
|
45
|
+
# - Codex: SDK agent creation
|
|
46
|
+
# - Others: file-based prompt handoff
|
|
50
47
|
```
|
|
51
48
|
|
|
52
49
|
### Context Propagation
|
|
@@ -127,13 +124,13 @@ A subagent can itself become an orchestrator, spawning further subagents for com
|
|
|
127
124
|
│ ORCHESTRATOR │ Level 0: Main workflow
|
|
128
125
|
│ (ct-orchestrator) │
|
|
129
126
|
└─────────┬───────────┘
|
|
130
|
-
│
|
|
127
|
+
│ orchestrate.spawn
|
|
131
128
|
▼
|
|
132
129
|
┌─────────────────────┐
|
|
133
130
|
│ SUB-ORCHESTRATOR │ Level 1: Epic decomposition
|
|
134
131
|
│ (ct-epic-architect) │
|
|
135
132
|
└─────────┬───────────┘
|
|
136
|
-
│
|
|
133
|
+
│ orchestrate.spawn
|
|
137
134
|
▼
|
|
138
135
|
┌─────────────────────┐
|
|
139
136
|
│ WORKER AGENT │ Level 2: Task execution
|
|
@@ -184,13 +181,13 @@ Detailed findings go to output files, not manifest or response:
|
|
|
184
181
|
|
|
185
182
|
### Rule 4: Token Injection (SHOULD)
|
|
186
183
|
|
|
187
|
-
|
|
184
|
+
All tokens are resolved before prompt injection via `orchestrate.spawn`:
|
|
188
185
|
|
|
189
|
-
```
|
|
186
|
+
```
|
|
190
187
|
# Required tokens for all subagents
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
188
|
+
TASK_ID # Current task identifier
|
|
189
|
+
DATE # Execution date (YYYY-MM-DD)
|
|
190
|
+
TOPIC_SLUG # URL-safe topic name
|
|
194
191
|
```
|
|
195
192
|
|
|
196
193
|
---
|
|
@@ -211,14 +208,14 @@ TI_TOPIC_SLUG # URL-safe topic name
|
|
|
211
208
|
|
|
212
209
|
Before spawning subagent:
|
|
213
210
|
- [ ] Identify appropriate skill for task type
|
|
214
|
-
- [ ] Prepare token context (
|
|
215
|
-
- [ ]
|
|
216
|
-
- [ ]
|
|
211
|
+
- [ ] Prepare token context (TASK_ID, DATE, TOPIC_SLUG)
|
|
212
|
+
- [ ] Use `orchestrate.spawn` to generate fully-resolved prompt
|
|
213
|
+
- [ ] Verify token resolution is complete (`tokenResolution.fullyResolved`)
|
|
217
214
|
|
|
218
215
|
Before chaining to another skill:
|
|
219
216
|
- [ ] Determine if skill shares context (same agent) or needs delegation
|
|
220
217
|
- [ ] For same-agent: Use `Skill(skill="name")` or `/skill-name`
|
|
221
|
-
- [ ] For new agent: Use
|
|
218
|
+
- [ ] For new agent: Use `orchestrate.spawn` with full protocol injection
|
|
222
219
|
|
|
223
220
|
Before completion:
|
|
224
221
|
- [ ] Verify manifest entry appended
|
|
@@ -231,7 +228,7 @@ Before completion:
|
|
|
231
228
|
|
|
232
229
|
| Skill | Demonstrates |
|
|
233
230
|
|-------|--------------|
|
|
234
|
-
| `ct-orchestrator` | Single-level spawning via
|
|
231
|
+
| `ct-orchestrator` | Single-level spawning via orchestrate.spawn |
|
|
235
232
|
| `ct-documentor` | Skill chaining (lookup → write → review) |
|
|
236
233
|
| `ct-epic-architect` | Potential multi-level orchestration |
|
|
237
234
|
|
|
@@ -71,15 +71,15 @@ CLEO supports 16 testing frameworks. Configure your project's testing setup in `
|
|
|
71
71
|
|
|
72
72
|
## Supported Frameworks
|
|
73
73
|
|
|
74
|
-
| Framework | Extension | Ecosystem |
|
|
75
|
-
|
|
76
|
-
|
|
|
77
|
-
| jest,
|
|
78
|
-
| node:test, deno, bun | .test.ts | Runtime built-ins |
|
|
79
|
-
| pytest | _test.py | Python |
|
|
80
|
-
| go | _test.go | Go |
|
|
81
|
-
| cargo | .rs | Rust |
|
|
82
|
-
| custom | varies | Any |
|
|
74
|
+
| Framework | Extension | Ecosystem | Notes |
|
|
75
|
+
|-----------|-----------|-----------|-------|
|
|
76
|
+
| vitest | .test.ts | Node.js | **Default for CLEO projects** |
|
|
77
|
+
| jest, playwright, cypress, mocha, ava, uvu, tap | .test.js/.ts | Node.js | |
|
|
78
|
+
| node:test, deno, bun | .test.ts | Runtime built-ins | |
|
|
79
|
+
| pytest | _test.py | Python | |
|
|
80
|
+
| go | _test.go | Go | |
|
|
81
|
+
| cargo | .rs | Rust | |
|
|
82
|
+
| custom | varies | Any | |
|
|
83
83
|
|
|
84
84
|
## Validation Gates
|
|
85
85
|
|
|
@@ -40,6 +40,8 @@ MCP is the **primary** entry point. Use `cleo_query` for reads and `cleo_mutate`
|
|
|
40
40
|
| `system` | `context` | Context window usage |
|
|
41
41
|
| `skills` | `list` | Available skills |
|
|
42
42
|
| `skills` | `show` | Skill details (`params: { name }`) |
|
|
43
|
+
| `pipeline` | `stage.status` | Pipeline stage for epic (`params: { epicId }`) |
|
|
44
|
+
| `pipeline` | `stage.validate` | Validate gate before advancing (`params: { epicId, stage }`) |
|
|
43
45
|
|
|
44
46
|
### Key Write Operations (`cleo_mutate`)
|
|
45
47
|
|
|
@@ -54,6 +56,8 @@ MCP is the **primary** entry point. Use `cleo_query` for reads and `cleo_mutate`
|
|
|
54
56
|
| `tasks` | `start` | Start working on a task (`params: { taskId }`) |
|
|
55
57
|
| `research` | `link` | Link research to task (`params: { taskId, entryId }`) |
|
|
56
58
|
| `orchestrate` | `spawn` | Generate spawn prompt for subagent (`params: { taskId }`) |
|
|
59
|
+
| `pipeline` | `stage.record` | Record pipeline stage progress (`params: { epicId, stage, status }`) |
|
|
60
|
+
| `pipeline` | `stage.gate.pass` | Advance pipeline to next stage (`params: { epicId, stage }`) |
|
|
57
61
|
|
|
58
62
|
## CLI Fallback
|
|
59
63
|
|
|
@@ -180,6 +184,23 @@ After EVERY command:
|
|
|
180
184
|
|
|
181
185
|
**Lifecycle**: See CLEO-INJECTION.md standard tier for full RCASD-IVTR+C pipeline details, or `references/rcsd-lifecycle.md` for gate enforcement and subagent architecture.
|
|
182
186
|
|
|
187
|
+
## Pipeline Awareness
|
|
188
|
+
|
|
189
|
+
Epics follow the RCASD-IVTR+C lifecycle managed through pipeline stages. Use `pipeline.stage.status` to check where an epic is in its lifecycle:
|
|
190
|
+
|
|
191
|
+
| Stage | Purpose |
|
|
192
|
+
|-------|---------|
|
|
193
|
+
| `research` | Information gathering and analysis |
|
|
194
|
+
| `consensus` | Validate claims and decisions |
|
|
195
|
+
| `architecture_decision` | ADR and specification |
|
|
196
|
+
| `specification` | Formal requirements |
|
|
197
|
+
| `decomposition` | Task breakdown |
|
|
198
|
+
| `implementation` | Build functionality |
|
|
199
|
+
| `validation` | Verify against criteria |
|
|
200
|
+
| `testing` | Test coverage |
|
|
201
|
+
| `release` | Version and publish |
|
|
202
|
+
| `contribution` | Multi-agent consensus tracking |
|
|
203
|
+
|
|
183
204
|
## Time Estimates Prohibited
|
|
184
205
|
|
|
185
206
|
- **MUST NOT** estimate hours, days, weeks, or temporal duration
|
|
@@ -192,5 +213,4 @@ For detailed guidance on specific topics, see:
|
|
|
192
213
|
|
|
193
214
|
- **Session Protocol**: `references/session-protocol.md`
|
|
194
215
|
- **RCSD Lifecycle**: `references/rcsd-lifecycle.md`
|
|
195
|
-
- **Orchestrator Constraints**: `references/orchestrator-constraints.md`
|
|
196
216
|
- **Anti-Patterns**: `references/anti-patterns.md`
|
|
@@ -0,0 +1,214 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: ct-grade
|
|
3
|
+
description: Session grading for agent behavioral analysis. Use when evaluating agent session quality, running grade scenarios, or interpreting grade results. Triggers on grading tasks, session quality checks, or behavioral analysis needs.
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
tier: 2
|
|
6
|
+
core: false
|
|
7
|
+
category: quality
|
|
8
|
+
protocol: null
|
|
9
|
+
dependencies: []
|
|
10
|
+
sharedResources: []
|
|
11
|
+
compatibility:
|
|
12
|
+
- claude-code
|
|
13
|
+
- cursor
|
|
14
|
+
- windsurf
|
|
15
|
+
- gemini-cli
|
|
16
|
+
license: MIT
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
# Session Grading Guide
|
|
20
|
+
|
|
21
|
+
Session grading evaluates agent behavioral patterns against the CLEO protocol. It reads the audit log for a completed session and applies a 5-dimension rubric to produce a score (0-100), letter grade (A-F), and diagnostic flags.
|
|
22
|
+
|
|
23
|
+
## When to Use Grade Mode
|
|
24
|
+
|
|
25
|
+
Use grading when you need to:
|
|
26
|
+
- Evaluate how well an agent followed CLEO protocol during a session
|
|
27
|
+
- Identify behavioral anti-patterns (skipped discovery, missing session.end, etc.)
|
|
28
|
+
- Track improvement over time across multiple sessions
|
|
29
|
+
- Validate that orchestrated subagents followed protocol
|
|
30
|
+
|
|
31
|
+
Grading requires audit data. Sessions must be started with the `--grade` flag to enable audit log capture.
|
|
32
|
+
|
|
33
|
+
## Starting a Grade Session
|
|
34
|
+
|
|
35
|
+
### CLI
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
# Start a session with grading enabled
|
|
39
|
+
ct session start --scope epic:T001 --name "Feature work" --grade
|
|
40
|
+
|
|
41
|
+
# The --grade flag enables detailed audit logging
|
|
42
|
+
# All MCP and CLI operations are recorded for later analysis
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
### MCP
|
|
46
|
+
|
|
47
|
+
```
|
|
48
|
+
cleo_mutate({ domain: "session", operation: "start",
|
|
49
|
+
params: { scope: "epic:T001", name: "Feature work", grade: true }})
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
## Running Scenarios
|
|
53
|
+
|
|
54
|
+
The grading rubric evaluates 5 behavioral scenarios that map to protocol compliance:
|
|
55
|
+
|
|
56
|
+
### 1. Fresh Discovery
|
|
57
|
+
Tests whether the agent checks existing sessions and tasks before starting work. Evaluates `session.list` and `tasks.find` calls at session start.
|
|
58
|
+
|
|
59
|
+
### 2. Task Hygiene
|
|
60
|
+
Tests whether task creation follows protocol: descriptions provided, parent existence verified before subtask creation, no duplicate tasks.
|
|
61
|
+
|
|
62
|
+
### 3. Error Recovery
|
|
63
|
+
Tests whether the agent handles errors correctly: follows up `E_NOT_FOUND` with recovery lookups (`tasks.find` or `tasks.exists`), avoids duplicate creates after failures.
|
|
64
|
+
|
|
65
|
+
### 4. Full Lifecycle
|
|
66
|
+
Tests session discipline end-to-end: session listed before task ops, session properly ended, MCP-first usage patterns.
|
|
67
|
+
|
|
68
|
+
### 5. Multi-Domain Analysis
|
|
69
|
+
Tests progressive disclosure: use of `admin.help` or skill lookups, preference for `cleo_query` (MCP) over CLI for programmatic access.
|
|
70
|
+
|
|
71
|
+
## Evaluating Results
|
|
72
|
+
|
|
73
|
+
### CLI
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
# Grade a specific session
|
|
77
|
+
ct grade <sessionId>
|
|
78
|
+
|
|
79
|
+
# List all past grade results
|
|
80
|
+
ct grade --list
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### MCP
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
# Grade a session
|
|
87
|
+
cleo_query({ domain: "admin", operation: "grade",
|
|
88
|
+
params: { sessionId: "abc-123" }})
|
|
89
|
+
|
|
90
|
+
# List past grades
|
|
91
|
+
cleo_query({ domain: "admin", operation: "grade.list" })
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## Understanding the 5 Dimensions
|
|
95
|
+
|
|
96
|
+
Each dimension scores 0-20 points, totaling 0-100.
|
|
97
|
+
|
|
98
|
+
### S1: Session Discipline (20 pts)
|
|
99
|
+
|
|
100
|
+
| Points | Criteria |
|
|
101
|
+
|--------|----------|
|
|
102
|
+
| 10 | `session.list` called before first task operation |
|
|
103
|
+
| 10 | `session.end` called when work is complete |
|
|
104
|
+
|
|
105
|
+
**What it measures**: Does the agent check existing sessions before starting, and properly close sessions when done?
|
|
106
|
+
|
|
107
|
+
### S2: Discovery Efficiency (20 pts)
|
|
108
|
+
|
|
109
|
+
| Points | Criteria |
|
|
110
|
+
|--------|----------|
|
|
111
|
+
| 0-15 | `find:list` ratio >= 80% earns full 15; scales linearly below |
|
|
112
|
+
| 5 | `tasks.show` used for detail retrieval |
|
|
113
|
+
|
|
114
|
+
**What it measures**: Does the agent prefer `tasks.find` (low context cost) over `tasks.list` (high context cost) for discovery?
|
|
115
|
+
|
|
116
|
+
### S3: Task Hygiene (20 pts)
|
|
117
|
+
|
|
118
|
+
Starts at 20 and deducts for violations:
|
|
119
|
+
|
|
120
|
+
| Deduction | Violation |
|
|
121
|
+
|-----------|-----------|
|
|
122
|
+
| -5 each | `tasks.add` without a description |
|
|
123
|
+
| -3 | Subtasks created without `tasks.exists` parent check |
|
|
124
|
+
|
|
125
|
+
**What it measures**: Does the agent create well-formed tasks with descriptions and verify parents before creating subtasks?
|
|
126
|
+
|
|
127
|
+
### S4: Error Protocol (20 pts)
|
|
128
|
+
|
|
129
|
+
Starts at 20 and deducts for violations:
|
|
130
|
+
|
|
131
|
+
| Deduction | Violation |
|
|
132
|
+
|-----------|-----------|
|
|
133
|
+
| -5 each | `E_NOT_FOUND` error not followed by recovery lookup within 5 ops |
|
|
134
|
+
| -5 | Duplicate task creates detected (same title in session) |
|
|
135
|
+
|
|
136
|
+
**What it measures**: Does the agent recover gracefully from errors and avoid creating duplicate tasks?
|
|
137
|
+
|
|
138
|
+
### S5: Progressive Disclosure Use (20 pts)
|
|
139
|
+
|
|
140
|
+
| Points | Criteria |
|
|
141
|
+
|--------|----------|
|
|
142
|
+
| 10 | `admin.help` or skill lookup calls made |
|
|
143
|
+
| 10 | `cleo_query` (MCP gateway) used for programmatic access |
|
|
144
|
+
|
|
145
|
+
**What it measures**: Does the agent use progressive disclosure (help/skills) and prefer MCP over CLI?
|
|
146
|
+
|
|
147
|
+
## Interpreting Scores
|
|
148
|
+
|
|
149
|
+
### Letter Grades
|
|
150
|
+
|
|
151
|
+
| Grade | Score Range | Meaning |
|
|
152
|
+
|-------|-----------|---------|
|
|
153
|
+
| **A** | 90-100 | Excellent protocol adherence. Agent follows all best practices. |
|
|
154
|
+
| **B** | 75-89 | Good. Minor gaps in one or two dimensions. |
|
|
155
|
+
| **C** | 60-74 | Acceptable. Several protocol violations need attention. |
|
|
156
|
+
| **D** | 45-59 | Below expectations. Significant anti-patterns present. |
|
|
157
|
+
| **F** | 0-44 | Failing. Major protocol violations across multiple dimensions. |
|
|
158
|
+
|
|
159
|
+
### Reading the Output
|
|
160
|
+
|
|
161
|
+
The grade result includes:
|
|
162
|
+
- **score/maxScore**: Raw numeric score (e.g., `85/100`)
|
|
163
|
+
- **percent**: Percentage score
|
|
164
|
+
- **grade**: Letter grade (A-F)
|
|
165
|
+
- **dimensions**: Per-dimension breakdown with score, max, and evidence
|
|
166
|
+
- **flags**: Specific violations or improvement suggestions
|
|
167
|
+
- **entryCount**: Number of audit entries analyzed
|
|
168
|
+
|
|
169
|
+
### Flags
|
|
170
|
+
|
|
171
|
+
Flags are actionable diagnostic messages. Each flag identifies a specific behavioral issue:
|
|
172
|
+
|
|
173
|
+
- `session.list never called` -- Check existing sessions before starting new ones
|
|
174
|
+
- `session.end never called` -- Always end sessions when done
|
|
175
|
+
- `tasks.list used Nx` -- Prefer `tasks.find` for discovery
|
|
176
|
+
- `tasks.add without description` -- Always provide task descriptions
|
|
177
|
+
- `Subtasks created without tasks.exists parent check` -- Verify parent exists first
|
|
178
|
+
- `E_NOT_FOUND not followed by recovery lookup` -- Follow errors with `tasks.find` or `tasks.exists`
|
|
179
|
+
- `No admin.help or skill lookup calls` -- Load `ct-cleo` for protocol guidance
|
|
180
|
+
- `No MCP query calls` -- Prefer `cleo_query` over CLI
|
|
181
|
+
|
|
182
|
+
## Common Anti-patterns
|
|
183
|
+
|
|
184
|
+
| Anti-pattern | Impact | Fix |
|
|
185
|
+
|-------------|--------|-----|
|
|
186
|
+
| Skipping `session.list` at start | -10 S1 | Always check existing sessions first |
|
|
187
|
+
| Forgetting `session.end` | -10 S1 | End sessions when work is complete |
|
|
188
|
+
| Using `tasks.list` instead of `tasks.find` | -up to 15 S2 | Use `find` for discovery, `list` only for known parent children |
|
|
189
|
+
| Creating tasks without descriptions | -5 each S3 | Always provide a description with `tasks.add` |
|
|
190
|
+
| Ignoring `E_NOT_FOUND` errors | -5 each S4 | Follow up with `tasks.find` or `tasks.exists` |
|
|
191
|
+
| Creating duplicate tasks | -5 S4 | Check for existing tasks before creating new ones |
|
|
192
|
+
| Never using `admin.help` | -10 S5 | Use progressive disclosure for protocol guidance |
|
|
193
|
+
| CLI-only usage (no MCP) | -10 S5 | Prefer `cleo_query`/`cleo_mutate` for programmatic access |
|
|
194
|
+
|
|
195
|
+
## Grade Result Schema
|
|
196
|
+
|
|
197
|
+
Grade results are stored in `.cleo/metrics/GRADES.jsonl` as append-only JSONL. Each entry conforms to `schemas/grade.schema.json` with these fields:
|
|
198
|
+
|
|
199
|
+
- `sessionId` (string, required) -- Session that was graded
|
|
200
|
+
- `taskId` (string, optional) -- Associated task ID
|
|
201
|
+
- `totalScore` (number, 0-100) -- Aggregate score
|
|
202
|
+
- `maxScore` (number, default 100) -- Maximum possible score
|
|
203
|
+
- `dimensions` (object) -- Per-dimension `{ score, max, evidence[] }`
|
|
204
|
+
- `flags` (string[]) -- Specific violations or suggestions
|
|
205
|
+
- `timestamp` (ISO 8601) -- When the grade was computed
|
|
206
|
+
- `entryCount` (number) -- Audit entries analyzed
|
|
207
|
+
- `evaluator` (`auto` | `manual`) -- How the grade was computed
|
|
208
|
+
|
|
209
|
+
## MCP Operations
|
|
210
|
+
|
|
211
|
+
| Gateway | Domain | Operation | Description |
|
|
212
|
+
|---------|--------|-----------|-------------|
|
|
213
|
+
| `cleo_query` | `admin` | `grade` | Grade a session (`params: { sessionId }`) |
|
|
214
|
+
| `cleo_query` | `admin` | `grade.list` | List all past grade results |
|