@neikyun/ciel 6.11.3 → 6.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/assets/.claude/agents/ciel-critic.md +71 -12
- package/assets/.claude/agents/ciel-explorer.md +59 -18
- package/assets/.claude/agents/ciel-improver.md +6 -3
- package/assets/.claude/agents/ciel-researcher.md +85 -25
- package/assets/.claude/hooks/block-destructive.sh +2 -2
- package/assets/.claude/hooks/check-test-first.sh +2 -2
- package/assets/.claude/hooks/memory-bootstrap.sh +0 -0
- package/assets/.claude/hooks/memory-engine.py +82 -15
- package/assets/.claude/hooks/post-tool-write.sh +32 -0
- package/assets/.claude/hooks/pre-agent-gate.sh +11 -6
- package/assets/.claude/hooks/pre-compact.sh +18 -0
- package/assets/.claude/hooks/pre-tool-write.sh +56 -31
- package/assets/.claude/hooks/session-start.sh +22 -1
- package/assets/.claude/hooks/session-version-check.sh +1 -1
- package/assets/.claude/hooks/stop.sh +104 -0
- package/assets/.claude/hooks/subagent-stop.sh +54 -0
- package/assets/.claude/hooks/track-file.sh +2 -2
- package/assets/.claude/hooks/user-prompt-submit.sh +11 -15
- package/assets/.claude/settings.json +18 -4
- package/assets/AGENTS.md +1 -1
- package/assets/CLAUDE.md +103 -175
- package/assets/commands/ciel-audit.md +58 -399
- package/assets/commands/ciel-create-skill.md +24 -38
- package/assets/commands/ciel-eval.md +25 -37
- package/assets/commands/ciel-init.md +36 -126
- package/assets/commands/ciel-status.md +22 -19
- package/assets/commands/ciel-update.md +20 -39
- package/assets/platforms/opencode/.opencode/agents/ciel-researcher.md +71 -895
- package/assets/platforms/opencode/.opencode/commands/ciel-audit.md +58 -296
- package/assets/platforms/opencode/.opencode/commands/ciel-create-skill.md +24 -46
- package/assets/platforms/opencode/.opencode/commands/ciel-eval.md +25 -45
- package/assets/platforms/opencode/.opencode/commands/ciel-init.md +36 -131
- package/assets/platforms/opencode/.opencode/commands/ciel-status.md +22 -24
- package/assets/platforms/opencode/.opencode/commands/ciel-update.md +20 -40
- package/assets/platforms/opencode/AGENTS.md +4 -4
- package/assets/rules/security.md +30 -0
- package/assets/rules/testing.md +23 -0
- package/assets/skills/agile/SKILL.md +42 -0
- package/assets/skills/alerting/SKILL.md +55 -0
- package/assets/skills/api-design/SKILL.md +46 -0
- package/assets/skills/appsec/SKILL.md +43 -0
- package/assets/skills/architecture/SKILL.md +74 -0
- package/assets/skills/backend/SKILL.md +41 -0
- package/assets/skills/backup-recovery/SKILL.md +42 -0
- package/assets/skills/caching/SKILL.md +44 -0
- package/assets/skills/cdn/SKILL.md +42 -0
- package/assets/skills/chaos/SKILL.md +41 -0
- package/assets/skills/cicd-pipeline/SKILL.md +56 -0
- package/assets/skills/cloud/SKILL.md +42 -0
- package/assets/skills/code-quality/SKILL.md +42 -0
- package/assets/skills/code-review/SKILL.md +41 -0
- package/assets/skills/communication/SKILL.md +42 -0
- package/assets/skills/containers/SKILL.md +42 -0
- package/assets/skills/cqrs/SKILL.md +41 -0
- package/assets/skills/crypto/SKILL.md +46 -0
- package/assets/skills/data-engineering/SKILL.md +42 -0
- package/assets/skills/database-design/SKILL.md +46 -0
- package/assets/skills/ddd/SKILL.md +45 -0
- package/assets/skills/deployment-strategies/SKILL.md +51 -0
- package/assets/skills/desktop/SKILL.md +42 -0
- package/assets/skills/devsecops/SKILL.md +43 -0
- package/assets/skills/event-driven/SKILL.md +46 -0
- package/assets/skills/frontend/SKILL.md +41 -0
- package/assets/skills/functional/SKILL.md +42 -0
- package/assets/skills/high-availability/SKILL.md +42 -0
- package/assets/skills/iac/SKILL.md +46 -0
- package/assets/skills/logging/SKILL.md +46 -0
- package/assets/skills/meta/ciel-improve/SKILL.md +127 -0
- package/assets/skills/meta/learnings-capture/SKILL.md +105 -0
- package/assets/skills/meta/patch-spec/patch-spec.md +50 -0
- package/assets/skills/meta/skill-creator/SKILL.md +115 -0
- package/assets/skills/meta/skill-freshness-auditor/SKILL.md +164 -0
- package/assets/skills/meta/skill-variant-evaluator/SKILL.md +100 -0
- package/assets/skills/meta/skills-first-design-auditor/SKILL.md +192 -0
- package/assets/skills/ml-engineering/SKILL.md +42 -0
- package/assets/skills/mobile/SKILL.md +42 -0
- package/assets/skills/monitoring/SKILL.md +54 -0
- package/assets/skills/networking/SKILL.md +42 -0
- package/assets/skills/nosql/SKILL.md +41 -0
- package/assets/skills/oop-solid/SKILL.md +42 -0
- package/assets/skills/performance/SKILL.md +41 -0
- package/assets/skills/reactive/SKILL.md +42 -0
- package/assets/skills/release-management/SKILL.md +51 -0
- package/assets/skills/research/fact-check-claims/SKILL.md +98 -0
- package/assets/skills/research/research-forums/SKILL.md +103 -0
- package/assets/skills/research/research-github-issues/SKILL.md +103 -0
- package/assets/skills/research/research-web-sources/SKILL.md +108 -0
- package/assets/skills/research/synthesize-findings/SKILL.md +112 -0
- package/assets/skills/research/validate-source-credibility/SKILL.md +103 -0
- package/assets/skills/resilience/SKILL.md +41 -0
- package/assets/skills/serverless/SKILL.md +42 -0
- package/assets/skills/servers/SKILL.md +41 -0
- package/assets/skills/sql/SKILL.md +45 -0
- package/assets/skills/supply-chain/SKILL.md +41 -0
- package/assets/skills/system-design/SKILL.md +91 -0
- package/assets/skills/tech-leadership/SKILL.md +46 -0
- package/assets/skills/testing/SKILL.md +41 -0
- package/assets/skills/tracing/SKILL.md +36 -0
- package/assets/skills/utility/branch-cleaner/SKILL.md +195 -0
- package/assets/skills/utility/branch-setup/SKILL.md +144 -0
- package/assets/skills/utility/changelog-updater/SKILL.md +125 -0
- package/assets/skills/utility/commit-writer/SKILL.md +154 -0
- package/assets/skills/utility/issue-closer/SKILL.md +106 -0
- package/assets/skills/utility/issue-creator/SKILL.md +200 -0
- package/assets/skills/utility/pr-merger/SKILL.md +189 -0
- package/assets/skills/utility/pr-opener/SKILL.md +180 -0
- package/assets/skills/utility/release-publisher/SKILL.md +224 -0
- package/assets/skills/workflow/ciel-dev-process/SKILL.md +94 -0
- package/assets/skills/workflow/faire-gatekeeper/SKILL.md +3 -1
- package/assets/skills/workflow/prouver-verifier/SKILL.md +11 -2
- package/dist/cli/check.d.ts.map +1 -1
- package/dist/cli/check.js +11 -2
- package/dist/cli/check.js.map +1 -1
- package/dist/cli/claude.d.ts.map +1 -1
- package/dist/cli/claude.js +0 -2
- package/dist/cli/claude.js.map +1 -1
- package/dist/cli/init.d.ts.map +1 -1
- package/dist/cli/init.js +11 -2
- package/dist/cli/init.js.map +1 -1
- package/dist/cli/opencode.d.ts.map +1 -1
- package/dist/cli/opencode.js +2 -1
- package/dist/cli/opencode.js.map +1 -1
- package/package.json +1 -1
- package/assets/commands/ciel-migrate.md +0 -35
- package/assets/commands/ciel-refresh.md +0 -91
|
@@ -1,327 +1,89 @@
|
|
|
1
1
|
---
|
|
2
|
-
description: Audits the current
|
|
2
|
+
description: Audits the current session for Ciel v7 paradigm violations — dispatch discipline, hook activity, skill coverage, agent quality, memory health. Produces a structured report with Ciel Health Score (0-100). Creates GitHub Issue if score < 90. Hook-independent — works even when Ciel hooks are broken.
|
|
3
3
|
---
|
|
4
4
|
|
|
5
|
-
# /ciel-audit — Session
|
|
5
|
+
# /ciel-audit — Session Post-Mortem
|
|
6
6
|
|
|
7
|
-
*Generates a structured report of Ciel behavior violations observed in the current session
|
|
7
|
+
*Generates a structured report of Ciel behavior violations observed in the current session with a Ciel Health Score (0-100). If score < 90, creates a GitHub Issue on KaosKyun/Ciel.*
|
|
8
8
|
|
|
9
9
|
Usage: `/ciel-audit`
|
|
10
10
|
|
|
11
|
-
Runs inline
|
|
11
|
+
Runs inline. No agent dispatch. Works without hooks.
|
|
12
12
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
## Instructions to the model
|
|
16
|
-
|
|
17
|
-
You are auditing the **current conversation session** — the one you are participating in right now. Jump directly to the analysis. No preamble. No meta-commentary. No "I will now audit…".
|
|
18
|
-
|
|
19
|
-
### What to audit
|
|
20
|
-
|
|
21
|
-
Scan the session's tool-use history (your own prior turns). For each `/ciel <task>` invocation in this session, check the **eight dimensions** below. For each dimension, assign a severity and penalty score to calculate the final Ciel Health Score.
|
|
22
|
-
|
|
23
|
-
#### Dimension 1: Dispatch discipline (critical) — penalty up to -25
|
|
24
|
-
|
|
25
|
-
- Did the assistant emit a `Task(subagent_type="ciel-*")` within the **first 3 tool calls** after the `/ciel` prompt?
|
|
26
|
-
- If NO: count the inline `Bash` / `Read` / `Grep` / `Glob` / `WebSearch` / `WebFetch` calls emitted in the main session before any dispatch. This is the v2.1.5 anti-pattern documented in `skills/ciel/SKILL.md:146-156`.
|
|
27
|
-
- Exception: Trivial tasks (rename, typo, 1-line fix, docs-only) are allowed to run inline.
|
|
28
|
-
- Severity→score:
|
|
29
|
-
- Critical dispatch violation (Standard/Critical task with no Task()): **-25**
|
|
30
|
-
- High (delayed dispatch, >1 inline tool before dispatch): **-15**
|
|
31
|
-
- OK (Task() within first 3 tool calls): **0**
|
|
32
|
-
|
|
33
|
-
#### Dimension 2: Hook activity (critical) — penalty up to -25
|
|
34
|
-
|
|
35
|
-
Search the session transcript for strings that Ciel hooks would have injected:
|
|
36
|
-
|
|
37
|
-
- `"CIEL depth hint:"` — from `hooks/user-prompt-submit.sh:36`, injected via `additionalContext` on every UserPromptSubmit.
|
|
38
|
-
- `"CIEL "` prefix (e.g., `"CIEL [CRITIQUE]"`, `"CIEL src/...` ) — from `hooks/pre-tool-write.sh:34,36`, injected before every Write/Edit.
|
|
39
|
-
- Session banner from `hooks/session-start.sh` (SessionStart context).
|
|
40
|
-
- `"META-CRITIQUER"` or similar end-of-session signal from `hooks/stop.sh`.
|
|
41
|
-
|
|
42
|
-
- None found despite writes/edits: **-25** → hooks broken
|
|
43
|
-
- Partial (some hooks firing but not all): **-10**
|
|
44
|
-
- All present: **0**
|
|
45
|
-
|
|
46
|
-
Most likely root cause: relative paths in `Ciel/settings.json:8,19,31,42,54,65,76`. The `command` field is written as `bash .claude/plugins/ciel/hooks/<file>.sh`, which Claude Code resolves against the current working directory, not the plugin directory.
|
|
47
|
-
|
|
48
|
-
#### Dimension 3: Skill invocation coverage vs depth — penalty up to -15
|
|
49
|
-
|
|
50
|
-
Cross-reference `skills/ciel/SKILL.md:20-64` (Depth Gauge) with what actually happened:
|
|
51
|
-
|
|
52
|
-
- **Standard** task → `researcher` and `explorer` agents must be dispatched in parallel before FAIRE. Missing both: **-15**. Missing one: **-8**.
|
|
53
|
-
- **Critical** task → must additionally invoke `stride-analyzer` and `security-regression-check`. Missing: **-15**.
|
|
54
|
-
- Depth ambiguous and `depth-classifier` not invoked: **-5**.
|
|
55
|
-
|
|
56
|
-
#### Dimension 4: Skill overlap / redundancy — penalty up to -10
|
|
57
|
-
|
|
58
|
-
- Both `relire-critic` AND `critiquer-auditor` on the same diff: **-10**
|
|
59
|
-
- `meta-critiquer` did not fire at end-of-task: **-5**
|
|
60
|
-
- Same skill invoked 3+ times for same scope: **-5**
|
|
61
|
-
|
|
62
|
-
#### Dimension 5: Agent report quality — penalty up to -5
|
|
63
|
-
|
|
64
|
-
- Any `Task(subagent_type="ciel-*")` with returned message under 200 tokens: **-5** per occurrence (max -10)
|
|
13
|
+
## Audit Dimensions (10)
|
|
65
14
|
|
|
66
|
-
|
|
15
|
+
Score starts at 100. Subtract penalties for each violation found.
|
|
67
16
|
|
|
68
|
-
|
|
17
|
+
### D1: Dispatch discipline (up to -25)
|
|
18
|
+
Did the assistant dispatch `ciel-researcher` + `ciel-explorer` in parallel within the first 3 tool calls after the user prompt on Standard+ tasks?
|
|
19
|
+
- Missing both on Standard/Critical: **-25**
|
|
20
|
+
- Delayed dispatch (>1 inline tool before dispatch): **-15**
|
|
21
|
+
- Exception: Trivial tasks (rename, typo) allowed inline.
|
|
69
22
|
|
|
70
|
-
|
|
23
|
+
### D2: Hook activity (up to -25)
|
|
24
|
+
Search session transcript for Ciel hook injections:
|
|
25
|
+
- `"CIEL depth hint:"` (UserPromptSubmit)
|
|
26
|
+
- `"CIEL"` prefix on pre-write warnings
|
|
27
|
+
- SessionStart banner
|
|
28
|
+
- None found despite writes/edits: **-25** (hooks broken)
|
|
29
|
+
- Partial (some firing): **-10**
|
|
71
30
|
|
|
72
|
-
|
|
31
|
+
### D3: Skill coverage vs depth (up to -15)
|
|
32
|
+
- Standard task: missing researcher+explorer dispatch: **-15** each
|
|
33
|
+
- Critical task: missing stride-analyzer: **-15**
|
|
34
|
+
- Depth ambiguous + depth-classifier not invoked: **-5**
|
|
73
35
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
```
|
|
36
|
+
### D4: Skill overlap / redundancy (up to -10)
|
|
37
|
+
- Both relire-critic AND critiquer-auditor on same diff: **-10**
|
|
38
|
+
- meta-critiquer did not fire at end-of-task: **-5**
|
|
78
39
|
|
|
79
|
-
|
|
80
|
-
-
|
|
81
|
-
- Versions match or local is newer: **0**
|
|
40
|
+
### D5: Agent report quality (up to -10)
|
|
41
|
+
- Any agent return under 200 tokens: **-5** per occurrence (max -10)
|
|
82
42
|
|
|
83
|
-
|
|
84
|
-
|
|
43
|
+
### D6: Intent routing (up to -10)
|
|
44
|
+
Cross-reference user prompt intent signals with skills invoked. Each miss: **-5** (max -10).
|
|
85
45
|
|
|
86
|
-
|
|
46
|
+
### D7: Version staleness (up to -10)
|
|
47
|
+
Check npm for newer `@neikyun/ciel` version. Remote > local: **-10**.
|
|
87
48
|
|
|
88
|
-
|
|
49
|
+
### D8: Platform health (up to -5)
|
|
50
|
+
Verify Claude Code (3+ agents, session-start.sh, settings.json) and OpenCode (plugin, 3+ agents, 5+ commands). Missing platform: **-3** to **-5**.
|
|
89
51
|
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
CLAUDE_SETTINGS=$(test -f .claude/settings.json && echo "1" || echo "0")
|
|
95
|
-
echo "Claude: agents=$CLAUDE_AGENTS hook=$CLAUDE_HOOK settings=$CLAUDE_SETTINGS"
|
|
96
|
-
if [ "$CLAUDE_AGENTS" -ge 3 ] && [ "$CLAUDE_HOOK" = "1" ] && [ "$CLAUDE_SETTINGS" = "1" ]; then
|
|
97
|
-
echo "Claude platform: OK"
|
|
98
|
-
else
|
|
99
|
-
echo "Claude platform: INCOMPLETE"
|
|
100
|
-
fi
|
|
101
|
-
```
|
|
102
|
-
Expected: at least 3 agent files + session-start.sh + settings.json
|
|
52
|
+
### D9: Memory health (up to -15)
|
|
53
|
+
- `.ciel/memory/index.json` missing: **-10**
|
|
54
|
+
- Episodes empty: **-5**
|
|
55
|
+
- Auto-memory contamination (MEMORY.md newer than Ciel episodes): **-5**
|
|
103
56
|
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
OPENCODE_PLUGIN=$(test -f .opencode/plugins/ciel.ts && echo "1" || echo "0")
|
|
107
|
-
OPENCODE_AGENTS=$(ls .opencode/agents/ciel-*.md 2>/dev/null | wc -l | tr -d ' ')
|
|
108
|
-
OPENCODE_COMMANDS=$(ls .opencode/commands/ciel*.md 2>/dev/null | wc -l | tr -d ' ')
|
|
109
|
-
echo "OpenCode: plugin=$OPENCODE_PLUGIN agents=$OPENCODE_AGENTS commands=$OPENCODE_COMMANDS"
|
|
110
|
-
if [ "$OPENCODE_PLUGIN" = "1" ] && [ "$OPENCODE_AGENTS" -ge 3 ] && [ "$OPENCODE_COMMANDS" -ge 5 ]; then
|
|
111
|
-
echo "OpenCode platform: OK"
|
|
112
|
-
else
|
|
113
|
-
echo "OpenCode platform: INCOMPLETE"
|
|
114
|
-
fi
|
|
115
|
-
```
|
|
116
|
-
Expected: ciel.ts plugin + at least 3 agent files + at least 5 command files
|
|
57
|
+
### D10: Memory insight quality (up to -10)
|
|
58
|
+
Run `python3 .claude/hooks/memory-engine.py analyze`. Check: promotion candidates, dead anchors, recursion drift, tag explosion.
|
|
117
59
|
|
|
118
|
-
Scoring
|
|
119
|
-
- Both platforms fully present and valid: **0**
|
|
120
|
-
- One platform missing or incomplete: **-3**
|
|
121
|
-
- Both platforms missing or critically incomplete: **-5**
|
|
122
|
-
|
|
123
|
-
**Important**: Do NOT check for `.claude/plugins/ciel/platforms/` or `.opencode/platforms/` directories — these are not part of the v6 architecture. Platform files are installed directly into `.claude/` and `.opencode/` respectively. Do NOT check for codex, cursor, kilocode, lmstudio, ollama, or windsurf — these platforms are not yet implemented.
|
|
124
|
-
|
|
125
|
-
---
|
|
126
|
-
|
|
127
|
-
### Scoring
|
|
60
|
+
## Scoring
|
|
128
61
|
|
|
129
62
|
**Ciel Health Score** = 100 - sum(penalties)
|
|
130
63
|
|
|
131
|
-
| Score
|
|
132
|
-
|
|
64
|
+
| Score | Status | Issue created? |
|
|
65
|
+
|-------|--------|----------------|
|
|
133
66
|
| 90-100 | Excellent | No |
|
|
134
|
-
| 0-89 | Needs improvement |
|
|
135
|
-
|
|
136
|
-
The score is calculated automatically from the detected violations.
|
|
137
|
-
|
|
138
|
-
---
|
|
139
|
-
|
|
140
|
-
### Report format
|
|
141
|
-
|
|
142
|
-
Begin the output with the literal line `# Ciel Session Audit Report`. End with the literal line `**End of audit report.**` on its own line.
|
|
143
|
-
|
|
144
|
-
```markdown
|
|
145
|
-
# Ciel Session Audit Report
|
|
67
|
+
| 0-89 | Needs improvement | Yes |
|
|
146
68
|
|
|
147
|
-
|
|
148
|
-
**Ciel Health Score**: <N>/100 — <Excellent|Good|Needs improvement|Critical>
|
|
149
|
-
**npm**: local v<X.Y.Z> | npm v<X.Y.Z> | <up-to-date|update available>
|
|
150
|
-
**Platforms**: claude ✓ opencode ✓ (or ✗ if missing)
|
|
151
|
-
**Session summary**: <N> /ciel invocation(s), <N> total tool calls, <N> Task() dispatches, <N> inline Bash/Read/Grep/WebSearch calls in main session.
|
|
69
|
+
## Report Format
|
|
152
70
|
|
|
153
|
-
|
|
71
|
+
Output MUST start with `# Ciel Session Audit Report` and end with `**End of audit report.**`.
|
|
154
72
|
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
## Violations detected
|
|
158
|
-
|
|
159
|
-
### <N>. <Short violation name> — penalty: -<N>
|
|
160
|
-
|
|
161
|
-
**Severity**: <critical | high | medium | low>
|
|
162
|
-
**Evidence from session**
|
|
163
|
-
- User prompt (turn N): `<exact prompt text, truncated to 200 chars>`
|
|
164
|
-
- Assistant's first 3 tool calls after this prompt:
|
|
165
|
-
1. `<tool_name>(<brief args>)`
|
|
166
|
-
2. `<tool_name>(<brief args>)`
|
|
167
|
-
3. `<tool_name>(<brief args>)`
|
|
168
|
-
- Expected: `Task(subagent_type="ciel-<role>", ...)` as first tool call
|
|
169
|
-
- Observed: `<actual behavior>`
|
|
170
|
-
|
|
171
|
-
**Root cause hypothesis**
|
|
172
|
-
<one or two sentences>
|
|
173
|
-
|
|
174
|
-
**Proposed fix**
|
|
175
|
-
- <concrete edit with file:line>
|
|
176
|
-
|
|
177
|
-
...
|
|
178
|
-
|
|
179
|
-
---
|
|
180
|
-
|
|
181
|
-
## Update notification
|
|
182
|
-
|
|
183
|
-
<If npm version > local version, show update notification here. Otherwise omit this section.>
|
|
184
|
-
|
|
185
|
-
---
|
|
186
|
-
|
|
187
|
-
## Scoring breakdown
|
|
188
|
-
|
|
189
|
-
| Dimension | Penalty |
|
|
190
|
-
|-----------|---------|
|
|
191
|
-
| D1 — Dispatch discipline | -<N> |
|
|
192
|
-
| D2 — Hook activity | -<N> |
|
|
193
|
-
| D3 — Skill coverage vs depth | -<N> |
|
|
194
|
-
| D4 — Skill overlap | -<N> |
|
|
195
|
-
| D5 — Agent report quality | -<N> |
|
|
196
|
-
| D6 — Intent routing | -<N> |
|
|
197
|
-
| D7 — npm version | -<N> |
|
|
198
|
-
| D8 — Platform health | -<N> |
|
|
199
|
-
| **Total** | **-<N>** |
|
|
200
|
-
| **Health Score** | **<N>/100** |
|
|
201
|
-
|
|
202
|
-
---
|
|
203
|
-
|
|
204
|
-
## Summary of fixes to apply
|
|
205
|
-
|
|
206
|
-
Apply these in order. Each points to a file:line and a concrete change.
|
|
207
|
-
|
|
208
|
-
1. **<Fix name>** — `<path>:<line>` — <one-line description>
|
|
209
|
-
2. ...
|
|
210
|
-
|
|
211
|
-
After each fix: re-run the scenario that triggered the violation in a fresh Claude session to verify.
|
|
212
|
-
|
|
213
|
-
---
|
|
214
|
-
|
|
215
|
-
**End of audit report.**
|
|
216
|
-
```
|
|
217
|
-
|
|
218
|
-
### If no violations are found (score = 100)
|
|
219
|
-
|
|
220
|
-
Output a single short section — **no issue is created** for PASS verdicts:
|
|
221
|
-
|
|
222
|
-
```markdown
|
|
223
|
-
# Ciel Session Audit Report
|
|
224
|
-
|
|
225
|
-
**Date**: <today's date>
|
|
226
|
-
**Ciel Health Score**: 100/100 — Excellent
|
|
227
|
-
**npm**: local v<X.Y.Z> | npm v<X.Y.Z> | up-to-date
|
|
228
|
-
**Platforms**: claude ✓ opencode ✓
|
|
229
|
-
**Session summary**: <N> /ciel invocation(s), <N> tool calls, <N> Task() dispatches.
|
|
230
|
-
**Verdict**: PASS
|
|
231
|
-
|
|
232
|
-
**No violations found.** No issue created.
|
|
233
|
-
|
|
234
|
-
**End of audit report.**
|
|
235
|
-
```
|
|
236
|
-
|
|
237
|
-
---
|
|
238
|
-
|
|
239
|
-
### GitHub Issue creation (only if score < 90)
|
|
240
|
-
|
|
241
|
-
If the Ciel Health Score is **below 90**, create a GitHub Issue with the report AND the session timeline.
|
|
242
|
-
|
|
243
|
-
**Important**: Do NOT create an issue if score >= 90. Only create for scores < 90.
|
|
244
|
-
|
|
245
|
-
1. **Check for duplicate issues first**:
|
|
246
|
-
```bash
|
|
247
|
-
EXISTING=$(gh issue list --repo KaosKyun/Ciel --label audit --state open \
|
|
248
|
-
--json title --jq '.[].title' 2>/dev/null | \
|
|
249
|
-
grep -c "^\[CIEL-AUDIT\] $(date +%Y-%m-%d) -" || true)
|
|
250
|
-
if [ "$EXISTING" -gt 0 ]; then
|
|
251
|
-
echo "Warning: today's audit issue already exists. Skipping creation."
|
|
252
|
-
echo "View existing at: https://github.com/KaosKyun/Ciel/issues?q=is%3Aopen+label%3Aaudit"
|
|
253
|
-
exit 0
|
|
254
|
-
fi
|
|
255
|
-
```
|
|
256
|
-
|
|
257
|
-
2. **Save the report to a temp file**: Write the complete report text to a temp file:
|
|
258
|
-
```bash
|
|
259
|
-
REPORT_FILE="/tmp/ciel-audit-report-$(date +%Y%m%d).md"
|
|
260
|
-
cat > "$REPORT_FILE" << 'REPORT_EOF'
|
|
261
|
-
# Ciel Session Audit Report
|
|
262
|
-
...
|
|
263
|
-
**End of audit report.**
|
|
264
|
-
REPORT_EOF
|
|
265
|
-
```
|
|
266
|
-
|
|
267
|
-
3. **Determine the repository**: run `gh repo view --json owner,name` to confirm, or default to `KaosKyun/Ciel`.
|
|
268
|
-
|
|
269
|
-
4. **Create the issue** with `gh issue create`, using Python to avoid shell quoting issues:
|
|
270
|
-
```bash
|
|
271
|
-
python3 -c "
|
|
272
|
-
import subprocess, sys
|
|
273
|
-
ymd, date_str, score, verdict = sys.argv[1:5]
|
|
274
|
-
report_path = f'/tmp/ciel-audit-report-{ymd}.md'
|
|
275
|
-
with open(report_path) as f:
|
|
276
|
-
body = f.read()
|
|
277
|
-
subprocess.run(['gh', 'issue', 'create',
|
|
278
|
-
'--repo', 'KaosKyun/Ciel',
|
|
279
|
-
'--title', f'[CIEL-AUDIT] {date_str} - {verdict} (Score: {score}/100)',
|
|
280
|
-
'--label', 'audit,ciel',
|
|
281
|
-
'--body', body])
|
|
282
|
-
" "$(date +%Y%m%d)" "$(date +%Y-%m-%d)" "<score>" "<verdict>"
|
|
283
|
-
```
|
|
284
|
-
|
|
285
|
-
5. **Include the session timeline** in the issue body. Before the report, add a timeline section listing all `/ciel` invocations in chronological order:
|
|
286
|
-
```markdown
|
|
287
|
-
## Session Timeline
|
|
288
|
-
|
|
289
|
-
| Time | Event |
|
|
290
|
-
|------|-------|
|
|
291
|
-
| T+N | /ciel <prompt excerpt> |
|
|
292
|
-
| T+N | Write/Edit on <file> |
|
|
293
|
-
| T+N | Task() dispatch to <agent> |
|
|
294
|
-
...
|
|
295
|
-
|
|
296
|
-
---
|
|
297
|
-
|
|
298
|
-
<full audit report>
|
|
299
|
-
```
|
|
300
|
-
|
|
301
|
-
6. **Handle errors gracefully**:
|
|
302
|
-
- If `gh` is not available: print a message telling the user to install (`brew install gh`) and print the report to stdout instead.
|
|
303
|
-
- If the repository can't be reached: skip issue creation, print a warning, but still output the report.
|
|
304
|
-
|
|
305
|
-
7. **After creating the issue**: include the issue URL at the end of your response so the user can open it directly.
|
|
306
|
-
|
|
307
|
-
**Title format**: `[CIEL-AUDIT] YYYY-MM-DD - VIOLATIONS FOUND (Score: <N>/100)`
|
|
308
|
-
|
|
309
|
-
**Labels**: `audit`, `ciel`
|
|
310
|
-
|
|
311
|
-
---
|
|
73
|
+
For violations: provide evidence (turn number, tool calls), root cause hypothesis (file:line), and proposed fix (concrete edit). No preamble, no meta-commentary, no emoji.
|
|
312
74
|
|
|
313
|
-
|
|
75
|
+
## GitHub Issue (score < 90 only)
|
|
314
76
|
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
77
|
+
1. Check for duplicate: `gh issue list --repo KaosKyun/Ciel --label audit --state open`
|
|
78
|
+
2. Create with title: `[CIEL-AUDIT] YYYY-MM-DD - VIOLATIONS FOUND (Score: N/100)`
|
|
79
|
+
3. Labels: `audit`, `ciel`
|
|
80
|
+
4. Body: session timeline + full audit report
|
|
81
|
+
5. Handle gracefully if `gh` unavailable — print report to stdout
|
|
318
82
|
|
|
319
|
-
|
|
83
|
+
## Constraints
|
|
320
84
|
|
|
321
|
-
- Do NOT fix violations in
|
|
322
|
-
- Do NOT
|
|
323
|
-
- Do NOT
|
|
324
|
-
- Do NOT
|
|
325
|
-
-
|
|
326
|
-
- Do NOT create duplicate issues — run the `gh issue list` check before creating.
|
|
327
|
-
- Do NOT restart, rerun, or attempt to fix the session in-flight. The audit report is the deliverable.
|
|
85
|
+
- Do NOT fix violations in-session. Report only.
|
|
86
|
+
- Do NOT dispatch agents. Inline only.
|
|
87
|
+
- Do NOT create issues for score >= 90.
|
|
88
|
+
- Do NOT create duplicate issues.
|
|
89
|
+
- Target length: 200-400 lines for violations, 50 for PASS.
|
|
@@ -1,50 +1,40 @@
|
|
|
1
1
|
---
|
|
2
|
-
description:
|
|
3
|
-
agent: ciel-improver
|
|
4
|
-
subtask: true
|
|
2
|
+
description: Generates a valid Ciel v7 SKILL.md scaffold — kebab-case name, YAML frontmatter with triggers.path, 3-section body (Checklist / Anti-patterns / Patterns), max 200 lines.
|
|
5
3
|
---
|
|
6
4
|
|
|
7
|
-
|
|
5
|
+
# /ciel-create-skill — Create a new Ciel v7 skill
|
|
8
6
|
|
|
9
|
-
|
|
10
|
-
description: Generates a valid Ciel SKILL.md scaffold following Anthropic Skills-first rules (kebab-case ≤64, YAML description ≤1536, body ≤500 lines).
|
|
11
|
-
---
|
|
12
|
-
|
|
13
|
-
# /ciel-create-skill — Create a new Ciel skill
|
|
14
|
-
|
|
15
|
-
*Generates a valid SKILL.md scaffold following Anthropic Skills-first rules (kebab-case name ≤64 chars, YAML frontmatter ≤1536-char description, ≤500-line body, progressive disclosure to one reference.md).*
|
|
7
|
+
Generates a valid SKILL.md scaffold following Ciel v7 format: 3 sections (Checklist, Anti-patterns, Patterns), kebab-case name, optional path trigger.
|
|
16
8
|
|
|
17
|
-
Usage: `/ciel-create-skill <name> <
|
|
9
|
+
Usage: `/ciel-create-skill <name> <domain-description>`
|
|
18
10
|
|
|
19
|
-
- `name` — kebab-case, max 64 chars, unique across
|
|
20
|
-
- `
|
|
11
|
+
- `name` — kebab-case, max 64 chars, unique across `.claude/skills/`
|
|
12
|
+
- `domain-description` — one-line description of what domain expertise this skill encodes
|
|
21
13
|
|
|
22
14
|
---
|
|
23
15
|
|
|
24
16
|
## What it does
|
|
25
17
|
|
|
26
|
-
1. Dispatches
|
|
27
|
-
2.
|
|
28
|
-
3.
|
|
29
|
-
4. Generates
|
|
30
|
-
5.
|
|
31
|
-
6.
|
|
18
|
+
1. Dispatches `ciel-improver` agent with MODE=CREATE-SKILL
|
|
19
|
+
2. Validates name (kebab-case, uniqueness, no reserved words)
|
|
20
|
+
3. Helps you define the path trigger (if any) and anti-patterns
|
|
21
|
+
4. Generates SKILL.md scaffold in v7 3-section format
|
|
22
|
+
5. On approval, writes to `.claude/skills/<name>/SKILL.md`
|
|
23
|
+
6. Optionally runs `sync-skills.sh` to distribute to mirrors
|
|
32
24
|
|
|
33
25
|
---
|
|
34
26
|
|
|
35
27
|
## Example
|
|
36
28
|
|
|
37
29
|
```
|
|
38
|
-
/ciel-create-skill kotlin-coroutines
|
|
30
|
+
/ciel-create-skill kotlin-coroutines "Kotlin coroutines — structured concurrency, Flow operators, cancellation, testing. Use when working with suspend functions, CoroutineScope, Flow."
|
|
39
31
|
```
|
|
40
32
|
|
|
41
33
|
Expected output:
|
|
42
|
-
1. Validation:
|
|
43
|
-
2.
|
|
44
|
-
3. Preview of `
|
|
45
|
-
4.
|
|
46
|
-
5. Catalog entry: `| kotlin-coroutines-mastery | Kotlin files with suspend/Flow |`
|
|
47
|
-
6. Approve? [y/n/edit]
|
|
34
|
+
1. Validation: name valid and unique
|
|
35
|
+
2. Path trigger suggestion: `**/*.kt,**/*.kts`
|
|
36
|
+
3. Preview of `SKILL.md` scaffold (~100-150 lines)
|
|
37
|
+
4. Approve? [y/n/edit]
|
|
48
38
|
|
|
49
39
|
---
|
|
50
40
|
|
|
@@ -53,33 +43,21 @@ Expected output:
|
|
|
53
43
|
- kebab-case only (no underscores, no camelCase)
|
|
54
44
|
- Max 64 chars
|
|
55
45
|
- No reserved words: `anthropic`, `claude`, `mcp`
|
|
56
|
-
- Must be unique across
|
|
57
|
-
- Warn if starts with category name (e.g. `workflow-foo` in `workflow/` is redundant)
|
|
58
|
-
|
|
59
|
-
---
|
|
60
|
-
|
|
61
|
-
## Category decision
|
|
62
|
-
|
|
63
|
-
- `workflow` — enforces a CRÉER/CRITIQUER/META-CRITIQUER step
|
|
64
|
-
- `research` — finds information outside the codebase
|
|
65
|
-
- `domain` — encodes expertise in a specific tech or pattern family
|
|
66
|
-
- `utility` — wraps a frequent mechanical operation
|
|
67
|
-
- `meta` — modifies Ciel itself
|
|
46
|
+
- Must be unique across `.claude/skills/`
|
|
68
47
|
|
|
69
48
|
---
|
|
70
49
|
|
|
71
50
|
## Guardrails
|
|
72
51
|
|
|
73
52
|
- **Max 1 new skill per invocation** — prevents skill explosion
|
|
74
|
-
- **SKILL.md
|
|
75
|
-
- **
|
|
76
|
-
- **
|
|
77
|
-
- **Never creates**: files in `.claude-plugin/`, agents, hooks, commands — those use different patterns
|
|
53
|
+
- **SKILL.md max 200 lines** — v7 hard cap
|
|
54
|
+
- **Duplication detection**: warns if description overlaps >= 70% with existing skill
|
|
55
|
+
- **Never creates**: agent definitions, hooks, commands — those use different patterns
|
|
78
56
|
|
|
79
57
|
---
|
|
80
58
|
|
|
81
59
|
## After creation
|
|
82
60
|
|
|
83
|
-
1. Test the skill
|
|
84
|
-
2.
|
|
85
|
-
3.
|
|
61
|
+
1. Test the skill on a relevant task — does it load when expected?
|
|
62
|
+
2. Commit with message: `feat(skills): add <name> skill`
|
|
63
|
+
3. Run `bash scripts/sync-skills.sh` to distribute to all mirrors
|
|
@@ -1,58 +1,42 @@
|
|
|
1
1
|
---
|
|
2
|
-
description:
|
|
3
|
-
agent: ciel-improver
|
|
4
|
-
subtask: true
|
|
2
|
+
description: Runs the binary eval dataset for a Ciel skill via headless Claude Code, comparing baseline scores and persisting results.
|
|
5
3
|
---
|
|
6
4
|
|
|
7
|
-
|
|
5
|
+
# /ciel-eval — Run skill eval harness
|
|
8
6
|
|
|
9
|
-
|
|
10
|
-
description: Runs the binary eval dataset for one or all Ciel skills via claude --print headless, comparing variants.
|
|
11
|
-
---
|
|
12
|
-
|
|
13
|
-
# /ciel-eval — Run eval harness
|
|
14
|
-
|
|
15
|
-
*Runs the binary eval dataset for one skill (or all skills) via `claude --print` headless mode, comparing variants and persisting scoreboards.*
|
|
7
|
+
Runs the binary eval dataset for a skill via headless Claude Code, persisting scoreboards to `~/.ciel/evals/results/`.
|
|
16
8
|
|
|
17
9
|
Usage: `/ciel-eval [skill-name]`
|
|
18
10
|
|
|
19
|
-
- No arg
|
|
20
|
-
- `skill-name`
|
|
11
|
+
- No arg — runs baseline eval for ALL skills that have a dataset
|
|
12
|
+
- `skill-name` — runs eval for that skill only
|
|
21
13
|
|
|
22
14
|
---
|
|
23
15
|
|
|
24
16
|
## What it does
|
|
25
17
|
|
|
26
|
-
1. Dispatches
|
|
27
|
-
2.
|
|
28
|
-
3. For each target skill:
|
|
18
|
+
1. Dispatches `ciel-improver` agent in MODE=EVAL
|
|
19
|
+
2. For each target skill:
|
|
29
20
|
- Loads dataset from `evals/datasets/<skill-name>.jsonl`
|
|
30
|
-
-
|
|
31
|
-
-
|
|
32
|
-
|
|
33
|
-
|
|
21
|
+
- Runs each entry via `claude --print` headless
|
|
22
|
+
- Scores against expected behaviors
|
|
23
|
+
3. Persists results to `~/.ciel/evals/results/<skill-name>-<timestamp>.json`
|
|
24
|
+
4. Returns a scoreboard
|
|
34
25
|
|
|
35
26
|
---
|
|
36
27
|
|
|
37
28
|
## Example output
|
|
38
29
|
|
|
39
30
|
```
|
|
40
|
-
# Eval results —
|
|
31
|
+
# Eval results — api-design
|
|
41
32
|
|
|
42
|
-
Dataset: evals/datasets/
|
|
43
|
-
Variants evaluated: 3
|
|
33
|
+
Dataset: evals/datasets/api-design.jsonl (12 entries)
|
|
44
34
|
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
| B (tightened) | 0.89 | 15.1k | 13.2s ← WINNER |
|
|
49
|
-
| C (reduced) | 0.81 | 12.8k | 11.7s |
|
|
35
|
+
Score: 0.85 (25/28 criteria passed)
|
|
36
|
+
Tokens used: 18.2k
|
|
37
|
+
Duration: 15.3s
|
|
50
38
|
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
Recommendation: adopt Variant B via `/ciel-improve`
|
|
54
|
-
|
|
55
|
-
Result persisted: evals/results/flux-narrator-2026-04-17-142345.json
|
|
39
|
+
Result persisted: ~/.ciel/evals/results/api-design-2026-05-23-142345.json
|
|
56
40
|
```
|
|
57
41
|
|
|
58
42
|
---
|
|
@@ -60,29 +44,25 @@ Result persisted: evals/results/flux-narrator-2026-04-17-142345.json
|
|
|
60
44
|
## Guardrails
|
|
61
45
|
|
|
62
46
|
- **Max 20 eval entries per dataset** — prevents runaway costs
|
|
63
|
-
- **
|
|
64
|
-
- **
|
|
65
|
-
- **
|
|
66
|
-
- **Graceful fallback** — if `claude --print` is unavailable, outputs manual eval prompts and waits for user input
|
|
47
|
+
- **Cost estimation** — displayed before starting; abort if > 500k tokens projected
|
|
48
|
+
- **Read-only**: `--disallowed-tools "Write Edit NotebookEdit"` to prevent side effects
|
|
49
|
+
- **Graceful fallback**: if `claude --print` unavailable, outputs manual eval prompts
|
|
67
50
|
|
|
68
51
|
---
|
|
69
52
|
|
|
70
53
|
## When to run
|
|
71
54
|
|
|
72
|
-
- Before and after
|
|
73
|
-
- On
|
|
74
|
-
- When creating a new skill — establish baseline
|
|
75
|
-
- When adding new eval criteria — re-score existing skills
|
|
55
|
+
- Before and after `/ciel-improve` — baseline vs improved
|
|
56
|
+
- On VERSION bump — regression sanity check
|
|
57
|
+
- When creating a new skill — establish baseline
|
|
76
58
|
|
|
77
59
|
---
|
|
78
60
|
|
|
79
61
|
## Eval dataset format
|
|
80
62
|
|
|
81
|
-
One JSON object per line (JSONL).
|
|
63
|
+
One JSON object per line (JSONL). Required fields:
|
|
82
64
|
|
|
83
65
|
- `id` — unique in dataset
|
|
84
66
|
- `input` — prompt / context passed to skill
|
|
85
|
-
- `expected_behavior` — object mapping criterion name
|
|
67
|
+
- `expected_behavior` — object mapping criterion name to boolean
|
|
86
68
|
- `skill` — which skill this tests
|
|
87
|
-
|
|
88
|
-
See `skills/meta/skill-variant-evaluator/reference.md` for the full schema and supported criterion evaluators.
|