selftune 0.2.9 → 0.2.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +35 -35
- package/apps/local-dashboard/dist/assets/index-BZVLv70T.js +16 -0
- package/apps/local-dashboard/dist/assets/{vendor-react-BQH_6WrG.js → vendor-react-BXP54cYo.js} +4 -4
- package/apps/local-dashboard/dist/assets/{vendor-table-dK1QMLq9.js → vendor-table-DTF_SXoy.js} +1 -1
- package/apps/local-dashboard/dist/assets/{vendor-ui-CO2mrx6e.js → vendor-ui-CWU0d1wd.js} +66 -66
- package/apps/local-dashboard/dist/index.html +15 -15
- package/bin/selftune.cjs +1 -1
- package/cli/selftune/activation-rules.ts +1 -0
- package/cli/selftune/alpha-upload/build-payloads.ts +18 -2
- package/cli/selftune/alpha-upload/stage-canonical.ts +94 -0
- package/cli/selftune/auth/device-code.ts +32 -0
- package/cli/selftune/auto-update.ts +12 -0
- package/cli/selftune/badge/badge.ts +1 -0
- package/cli/selftune/canonical-export.ts +5 -0
- package/cli/selftune/claude-agents.ts +154 -0
- package/cli/selftune/contribute/bundle.ts +1 -0
- package/cli/selftune/contribute/contribute.ts +1 -0
- package/cli/selftune/cron/setup.ts +2 -2
- package/cli/selftune/dashboard-server.ts +1 -0
- package/cli/selftune/eval/hooks-to-evals.ts +1 -0
- package/cli/selftune/eval/import-skillsbench.ts +1 -0
- package/cli/selftune/eval/synthetic-evals.ts +2 -3
- package/cli/selftune/eval/unit-test.ts +1 -0
- package/cli/selftune/evolution/deploy-proposal.ts +1 -0
- package/cli/selftune/evolution/evolve-body.ts +93 -6
- package/cli/selftune/evolution/evolve.ts +0 -1
- package/cli/selftune/evolution/propose-body.ts +3 -2
- package/cli/selftune/evolution/propose-routing.ts +3 -2
- package/cli/selftune/evolution/refine-body.ts +3 -2
- package/cli/selftune/export.ts +1 -0
- package/cli/selftune/grading/grade-session.ts +8 -0
- package/cli/selftune/hooks/auto-activate.ts +1 -0
- package/cli/selftune/hooks/evolution-guard.ts +1 -1
- package/cli/selftune/hooks/prompt-log.ts +1 -0
- package/cli/selftune/hooks/session-stop.ts +34 -40
- package/cli/selftune/hooks/skill-change-guard.ts +1 -0
- package/cli/selftune/hooks/skill-eval.ts +1 -1
- package/cli/selftune/index.ts +23 -14
- package/cli/selftune/ingestors/claude-replay.ts +1 -0
- package/cli/selftune/ingestors/codex-rollout.ts +1 -0
- package/cli/selftune/ingestors/codex-wrapper.ts +1 -0
- package/cli/selftune/ingestors/openclaw-ingest.ts +1 -0
- package/cli/selftune/ingestors/opencode-ingest.ts +1 -0
- package/cli/selftune/init.ts +121 -29
- package/cli/selftune/localdb/db.ts +1 -0
- package/cli/selftune/localdb/direct-write.ts +39 -0
- package/cli/selftune/localdb/materialize.ts +2 -0
- package/cli/selftune/localdb/queries.ts +53 -0
- package/cli/selftune/localdb/schema.ts +28 -0
- package/cli/selftune/normalization.ts +1 -0
- package/cli/selftune/observability.ts +1 -0
- package/cli/selftune/repair/skill-usage.ts +1 -0
- package/cli/selftune/routes/orchestrate-runs.ts +1 -0
- package/cli/selftune/routes/overview.ts +1 -0
- package/cli/selftune/routes/skill-report.ts +1 -0
- package/cli/selftune/sync.ts +30 -1
- package/cli/selftune/uninstall.ts +412 -0
- package/cli/selftune/utils/canonical-log.ts +2 -0
- package/cli/selftune/utils/jsonl.ts +1 -0
- package/cli/selftune/utils/llm-call.ts +131 -3
- package/cli/selftune/utils/skill-log.ts +1 -0
- package/cli/selftune/utils/transcript.ts +1 -0
- package/cli/selftune/utils/trigger-check.ts +1 -1
- package/cli/selftune/workflows/skill-md-writer.ts +5 -5
- package/cli/selftune/workflows/workflows.ts +1 -0
- package/package.json +37 -33
- package/packages/telemetry-contract/fixtures/golden.test.ts +1 -0
- package/packages/telemetry-contract/package.json +1 -1
- package/packages/telemetry-contract/src/schemas.ts +1 -0
- package/packages/telemetry-contract/tests/compatibility.test.ts +1 -0
- package/packages/ui/README.md +35 -34
- package/packages/ui/package.json +3 -3
- package/packages/ui/src/components/ActivityTimeline.tsx +49 -42
- package/packages/ui/src/components/EvidenceViewer.tsx +306 -182
- package/packages/ui/src/components/EvolutionTimeline.tsx +83 -72
- package/packages/ui/src/components/InfoTip.tsx +4 -3
- package/packages/ui/src/components/OrchestrateRunsPanel.tsx +60 -53
- package/packages/ui/src/components/section-cards.tsx +19 -24
- package/packages/ui/src/components/skill-health-grid.tsx +213 -193
- package/packages/ui/src/lib/constants.tsx +1 -0
- package/packages/ui/src/primitives/badge.tsx +12 -15
- package/packages/ui/src/primitives/button.tsx +7 -7
- package/packages/ui/src/primitives/card.tsx +15 -26
- package/packages/ui/src/primitives/checkbox.tsx +7 -8
- package/packages/ui/src/primitives/collapsible.tsx +5 -5
- package/packages/ui/src/primitives/dropdown-menu.tsx +45 -55
- package/packages/ui/src/primitives/label.tsx +6 -6
- package/packages/ui/src/primitives/select.tsx +28 -37
- package/packages/ui/src/primitives/table.tsx +17 -44
- package/packages/ui/src/primitives/tabs.tsx +14 -21
- package/packages/ui/src/primitives/tooltip.tsx +10 -22
- package/skill/SKILL.md +70 -57
- package/skill/Workflows/AlphaUpload.md +4 -4
- package/skill/Workflows/AutoActivation.md +11 -6
- package/skill/Workflows/Badge.md +22 -16
- package/skill/Workflows/Baseline.md +34 -36
- package/skill/Workflows/Composability.md +16 -11
- package/skill/Workflows/Contribute.md +26 -21
- package/skill/Workflows/Cron.md +23 -22
- package/skill/Workflows/Dashboard.md +32 -27
- package/skill/Workflows/Doctor.md +33 -27
- package/skill/Workflows/Evals.md +48 -47
- package/skill/Workflows/EvolutionMemory.md +31 -21
- package/skill/Workflows/Evolve.md +84 -82
- package/skill/Workflows/EvolveBody.md +58 -47
- package/skill/Workflows/Grade.md +16 -13
- package/skill/Workflows/ImportSkillsBench.md +9 -6
- package/skill/Workflows/Ingest.md +36 -21
- package/skill/Workflows/Initialize.md +108 -40
- package/skill/Workflows/Orchestrate.md +22 -16
- package/skill/Workflows/Replay.md +12 -7
- package/skill/Workflows/Rollback.md +13 -6
- package/skill/Workflows/Schedule.md +6 -6
- package/skill/Workflows/Sync.md +18 -11
- package/skill/Workflows/UnitTest.md +28 -17
- package/skill/Workflows/Watch.md +28 -21
- package/skill/agents/diagnosis-analyst.md +11 -0
- package/skill/agents/evolution-reviewer.md +15 -1
- package/skill/agents/integration-guide.md +10 -0
- package/skill/agents/pattern-analyst.md +12 -1
- package/skill/references/grading-methodology.md +23 -24
- package/skill/references/interactive-config.md +7 -7
- package/skill/references/invocation-taxonomy.md +22 -20
- package/skill/references/logs.md +14 -6
- package/skill/references/setup-patterns.md +4 -2
- package/.claude/agents/diagnosis-analyst.md +0 -156
- package/.claude/agents/evolution-reviewer.md +0 -180
- package/.claude/agents/integration-guide.md +0 -212
- package/.claude/agents/pattern-analyst.md +0 -160
- package/apps/local-dashboard/dist/assets/index-C4UYGWKr.js +0 -15
|
@@ -1,212 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: integration-guide
|
|
3
|
-
description: Guided interactive setup of selftune for specific project types with verified configuration.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Integration Guide
|
|
7
|
-
|
|
8
|
-
## Role
|
|
9
|
-
|
|
10
|
-
Guide users through setting up selftune for their specific project. Detect
|
|
11
|
-
project structure, generate appropriate configuration, install hooks, and
|
|
12
|
-
verify the setup is working end-to-end.
|
|
13
|
-
|
|
14
|
-
**Activate when the user says:**
|
|
15
|
-
- "set up selftune"
|
|
16
|
-
- "integrate selftune"
|
|
17
|
-
- "configure selftune for my project"
|
|
18
|
-
- "install selftune"
|
|
19
|
-
- "get selftune working"
|
|
20
|
-
- "selftune setup guide"
|
|
21
|
-
|
|
22
|
-
## Connection to Workflows
|
|
23
|
-
|
|
24
|
-
This agent is the deep-dive version of the Initialize workflow, spawned by
|
|
25
|
-
the main agent as a subagent when the project structure is complex.
|
|
26
|
-
|
|
27
|
-
**Connected workflows:**
|
|
28
|
-
- **Initialize** — for complex project structures (monorepos, multi-skill repos, mixed agent platforms), spawn this agent instead of running the basic init workflow
|
|
29
|
-
|
|
30
|
-
**When to spawn:** when the project has multiple SKILL.md files, multiple
|
|
31
|
-
packages or workspaces, mixed agent platforms (Claude + Codex), or any
|
|
32
|
-
structure where the standard `selftune init` needs project-specific guidance.
|
|
33
|
-
|
|
34
|
-
## Context
|
|
35
|
-
|
|
36
|
-
You need access to:
|
|
37
|
-
- The user's project root directory
|
|
38
|
-
- `~/.selftune/config.json` (may not exist yet)
|
|
39
|
-
- `~/.claude/settings.json` (for hook installation)
|
|
40
|
-
- `skill/settings_snippet.json` (hook configuration template)
|
|
41
|
-
- `skill/Workflows/Initialize.md` (full init workflow reference)
|
|
42
|
-
- `skill/Workflows/Doctor.md` (health check reference)
|
|
43
|
-
|
|
44
|
-
## Workflow
|
|
45
|
-
|
|
46
|
-
### Step 1: Detect project structure
|
|
47
|
-
|
|
48
|
-
Examine the workspace to determine the project type:
|
|
49
|
-
|
|
50
|
-
**Single-skill project:**
|
|
51
|
-
- One `SKILL.md` at or near the project root
|
|
52
|
-
- Typical for focused tools and utilities
|
|
53
|
-
|
|
54
|
-
**Multi-skill project:**
|
|
55
|
-
- Multiple `SKILL.md` files in separate directories
|
|
56
|
-
- Skills are independent but coexist in one repo
|
|
57
|
-
|
|
58
|
-
**Monorepo:**
|
|
59
|
-
- Multiple packages/projects with their own skill files
|
|
60
|
-
- May have shared configuration at the root level
|
|
61
|
-
|
|
62
|
-
**No skills yet:**
|
|
63
|
-
- No `SKILL.md` files found
|
|
64
|
-
- User needs to create skills before selftune can observe them
|
|
65
|
-
|
|
66
|
-
Report what you find and confirm with the user.
|
|
67
|
-
|
|
68
|
-
### Step 2: Check existing configuration
|
|
69
|
-
|
|
70
|
-
```bash
|
|
71
|
-
selftune doctor
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
If selftune is already installed, parse the doctor output:
|
|
75
|
-
- **All checks pass** — setup is complete, offer to run a health audit
|
|
76
|
-
- **Some checks fail** — fix the failing checks (see Step 6)
|
|
77
|
-
- **Command not found** — proceed to Step 3
|
|
78
|
-
|
|
79
|
-
### Step 3: Install the CLI
|
|
80
|
-
|
|
81
|
-
Check if selftune is on PATH:
|
|
82
|
-
|
|
83
|
-
```bash
|
|
84
|
-
which selftune
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
If not installed:
|
|
88
|
-
|
|
89
|
-
```bash
|
|
90
|
-
npm install -g selftune
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
Verify installation succeeded before continuing.
|
|
94
|
-
|
|
95
|
-
### Step 4: Initialize configuration
|
|
96
|
-
|
|
97
|
-
```bash
|
|
98
|
-
selftune init
|
|
99
|
-
```
|
|
100
|
-
|
|
101
|
-
Parse the output to confirm `~/.selftune/config.json` was created. Note the
|
|
102
|
-
detected `agent_type` and `cli_path`.
|
|
103
|
-
|
|
104
|
-
If the user is on a non-Claude agent platform:
|
|
105
|
-
- **Codex** — inform about `ingest wrap-codex` and `ingest codex` options
|
|
106
|
-
- **OpenCode** — inform about `ingest opencode` option
|
|
107
|
-
|
|
108
|
-
### Step 5: Install hooks
|
|
109
|
-
|
|
110
|
-
For **Claude Code** users, merge hook entries from `skill/settings_snippet.json`
|
|
111
|
-
into `~/.claude/settings.json`. Three hooks are required:
|
|
112
|
-
|
|
113
|
-
| Hook | Script | Purpose |
|
|
114
|
-
|------|--------|---------|
|
|
115
|
-
| `UserPromptSubmit` | `hooks/prompt-log.ts` | Log every user query |
|
|
116
|
-
| `PostToolUse` (Read) | `hooks/skill-eval.ts` | Track skill triggers |
|
|
117
|
-
| `Stop` | `hooks/session-stop.ts` | Capture session telemetry |
|
|
118
|
-
|
|
119
|
-
Derive script paths from `cli_path` in `~/.selftune/config.json`.
|
|
120
|
-
|
|
121
|
-
For **Codex**: use `selftune ingest wrap-codex` or `selftune ingest codex`.
|
|
122
|
-
For **OpenCode**: use `selftune ingest opencode`.
|
|
123
|
-
|
|
124
|
-
### Step 6: Verify with doctor
|
|
125
|
-
|
|
126
|
-
```bash
|
|
127
|
-
selftune doctor
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
All checks must pass. For any failures:
|
|
131
|
-
|
|
132
|
-
| Failed Check | Resolution |
|
|
133
|
-
|-------------|------------|
|
|
134
|
-
| Log files missing | Run a test session to generate initial entries |
|
|
135
|
-
| Logs not parseable | Inspect and fix corrupted log lines |
|
|
136
|
-
| Hooks not installed | Re-check settings.json merge from Step 5 |
|
|
137
|
-
| Hook scripts missing | Verify paths point to actual files on disk |
|
|
138
|
-
| Audit log invalid | Remove corrupted entries |
|
|
139
|
-
|
|
140
|
-
Re-run doctor after each fix until all checks pass.
|
|
141
|
-
|
|
142
|
-
### Step 7: Run a smoke test
|
|
143
|
-
|
|
144
|
-
Execute a test session and verify telemetry capture:
|
|
145
|
-
|
|
146
|
-
1. Run a simple query that should trigger a skill
|
|
147
|
-
2. Check `~/.claude/session_telemetry_log.jsonl` for the new entry
|
|
148
|
-
3. Check `~/.claude/skill_usage_log.jsonl` for the trigger event
|
|
149
|
-
4. Check `~/.claude/all_queries_log.jsonl` for the query log
|
|
150
|
-
|
|
151
|
-
```bash
|
|
152
|
-
selftune last
|
|
153
|
-
```
|
|
154
|
-
|
|
155
|
-
Verify the session appears in the output.
|
|
156
|
-
|
|
157
|
-
### Step 8: Configure project-specific settings
|
|
158
|
-
|
|
159
|
-
Based on the project type detected in Step 1:
|
|
160
|
-
|
|
161
|
-
**Single-skill:** No additional configuration needed.
|
|
162
|
-
|
|
163
|
-
**Multi-skill:** Verify each skill's `SKILL.md` has a unique `name` field
|
|
164
|
-
and non-overlapping trigger keywords.
|
|
165
|
-
|
|
166
|
-
**Monorepo:** Ensure hook paths are absolute (not relative) so they work
|
|
167
|
-
from any package directory.
|
|
168
|
-
|
|
169
|
-
### Step 9: Provide next steps
|
|
170
|
-
|
|
171
|
-
Tell the user what to do next based on their goals:
|
|
172
|
-
|
|
173
|
-
- **"I want to see how my skills are doing"** — run `selftune status`
|
|
174
|
-
- **"I want to improve a skill"** — run `selftune eval generate --skill <name>` then `selftune evolve --skill <name>`
|
|
175
|
-
- **"I want to grade a session"** — run `selftune grade --skill <name>`
|
|
176
|
-
|
|
177
|
-
## Commands
|
|
178
|
-
|
|
179
|
-
| Command | Purpose |
|
|
180
|
-
|---------|---------|
|
|
181
|
-
| `selftune init` | Bootstrap configuration |
|
|
182
|
-
| `selftune doctor` | Verify installation health |
|
|
183
|
-
| `selftune status` | Post-setup health check |
|
|
184
|
-
| `selftune last` | Verify telemetry capture |
|
|
185
|
-
| `selftune eval generate --list-skills` | Confirm skills are being tracked |
|
|
186
|
-
|
|
187
|
-
## Output
|
|
188
|
-
|
|
189
|
-
Produce a setup completion summary:
|
|
190
|
-
|
|
191
|
-
```markdown
|
|
192
|
-
## selftune Setup Complete
|
|
193
|
-
|
|
194
|
-
### Environment
|
|
195
|
-
- Agent: <claude / codex / opencode>
|
|
196
|
-
- Project type: <single-skill / multi-skill / monorepo>
|
|
197
|
-
- Skills detected: <list of skill names>
|
|
198
|
-
|
|
199
|
-
### Configuration
|
|
200
|
-
- Config: ~/.selftune/config.json [created / verified]
|
|
201
|
-
- Hooks: [installed / N/A for non-Claude agents]
|
|
202
|
-
- Doctor: [all checks pass / N failures — see below]
|
|
203
|
-
|
|
204
|
-
### Verification
|
|
205
|
-
- Telemetry capture: [working / not verified]
|
|
206
|
-
- Skill tracking: [working / not verified]
|
|
207
|
-
|
|
208
|
-
### Next Steps
|
|
209
|
-
1. [Primary recommended action]
|
|
210
|
-
2. [Secondary action]
|
|
211
|
-
3. [Optional action]
|
|
212
|
-
```
|
|
@@ -1,160 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: pattern-analyst
|
|
3
|
-
description: Cross-skill pattern analysis, trigger conflict detection, and optimization recommendations.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Pattern Analyst
|
|
7
|
-
|
|
8
|
-
## Role
|
|
9
|
-
|
|
10
|
-
Analyze patterns across all skills in the system. Detect trigger conflicts
|
|
11
|
-
where multiple skills compete for the same queries, find optimization
|
|
12
|
-
opportunities, and identify systemic issues affecting multiple skills.
|
|
13
|
-
|
|
14
|
-
**Activate when the user says:**
|
|
15
|
-
- "skill patterns"
|
|
16
|
-
- "conflicts between skills"
|
|
17
|
-
- "cross-skill analysis"
|
|
18
|
-
- "which skills overlap"
|
|
19
|
-
- "skill trigger conflicts"
|
|
20
|
-
- "optimize my skills"
|
|
21
|
-
|
|
22
|
-
## Connection to Workflows
|
|
23
|
-
|
|
24
|
-
This agent is spawned by the main agent as a subagent for deep cross-skill
|
|
25
|
-
analysis.
|
|
26
|
-
|
|
27
|
-
**Connected workflows:**
|
|
28
|
-
- **Composability** — when `selftune eval composability` identifies conflict candidates, spawn this agent for deeper investigation of trigger overlaps and resolution strategies
|
|
29
|
-
- **Evals** — when analyzing cross-skill patterns or systemwide undertriggering, spawn this agent to find optimization opportunities
|
|
30
|
-
|
|
31
|
-
**When to spawn:** when the user asks about conflicts between skills,
|
|
32
|
-
cross-skill optimization, or when composability scores indicate moderate-to-severe
|
|
33
|
-
conflicts (score > 0.3).
|
|
34
|
-
|
|
35
|
-
## Context
|
|
36
|
-
|
|
37
|
-
You need access to:
|
|
38
|
-
- `~/.claude/skill_usage_log.jsonl` — which skills triggered for which queries
|
|
39
|
-
- `~/.claude/all_queries_log.jsonl` — all queries including non-triggers
|
|
40
|
-
- `~/.claude/session_telemetry_log.jsonl` — session-level metrics per skill
|
|
41
|
-
- `~/.claude/evolution_audit_log.jsonl` — evolution history across skills
|
|
42
|
-
- All skill `SKILL.md` files in the workspace
|
|
43
|
-
|
|
44
|
-
## Workflow
|
|
45
|
-
|
|
46
|
-
### Step 1: Inventory all skills
|
|
47
|
-
|
|
48
|
-
```bash
|
|
49
|
-
selftune eval generate --list-skills
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
Parse the JSON output to get a complete list of skills with their query
|
|
53
|
-
counts and session counts. This is your working set.
|
|
54
|
-
|
|
55
|
-
### Step 2: Gather per-skill health
|
|
56
|
-
|
|
57
|
-
```bash
|
|
58
|
-
selftune status
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
Record each skill's pass rate, session count, and status flags. Identify
|
|
62
|
-
skills that are healthy vs. those showing warnings or regressions.
|
|
63
|
-
|
|
64
|
-
### Step 3: Collect SKILL.md descriptions
|
|
65
|
-
|
|
66
|
-
For each skill returned in Step 1, locate and read its `SKILL.md` file.
|
|
67
|
-
Extract:
|
|
68
|
-
- The `description` field from frontmatter
|
|
69
|
-
- Trigger keywords from the workflow routing table
|
|
70
|
-
- Negative examples (if present)
|
|
71
|
-
|
|
72
|
-
### Step 4: Detect trigger conflicts
|
|
73
|
-
|
|
74
|
-
Compare trigger keywords and description phrases across all skills. Flag:
|
|
75
|
-
- **Direct conflicts** — two skills list the same trigger keyword
|
|
76
|
-
- **Semantic overlaps** — different words with the same meaning (e.g.,
|
|
77
|
-
"presentation" in skill A, "slide deck" in skill B)
|
|
78
|
-
- **Negative gaps** — a skill's negative examples overlap with another
|
|
79
|
-
skill's positive triggers
|
|
80
|
-
|
|
81
|
-
### Step 5: Analyze query routing patterns
|
|
82
|
-
|
|
83
|
-
Read `skill_usage_log.jsonl` and group by query text. Look for:
|
|
84
|
-
- Queries that triggered multiple skills (conflict signal)
|
|
85
|
-
- Queries that triggered no skills despite matching a description (gap signal)
|
|
86
|
-
- Queries that triggered the wrong skill (misroute signal)
|
|
87
|
-
|
|
88
|
-
### Step 6: Cross-skill telemetry comparison
|
|
89
|
-
|
|
90
|
-
For each skill, pull stats:
|
|
91
|
-
|
|
92
|
-
```bash
|
|
93
|
-
selftune eval generate --skill <name> --stats
|
|
94
|
-
```
|
|
95
|
-
|
|
96
|
-
Compare across skills:
|
|
97
|
-
- **Error rates** — are some skills consistently failing?
|
|
98
|
-
- **Turn counts** — outlier skills may have process issues
|
|
99
|
-
- **Tool call patterns** — skills with similar patterns may be duplicates
|
|
100
|
-
|
|
101
|
-
### Step 7: Check evolution interactions
|
|
102
|
-
|
|
103
|
-
Read `~/.claude/evolution_audit_log.jsonl` for all skills. Look for:
|
|
104
|
-
- Evolution in one skill that caused regression in another
|
|
105
|
-
- Skills evolved in parallel that now conflict
|
|
106
|
-
- Rollbacks that correlate with another skill's evolution
|
|
107
|
-
|
|
108
|
-
### Step 8: Synthesize findings
|
|
109
|
-
|
|
110
|
-
Compile a cross-skill analysis report.
|
|
111
|
-
|
|
112
|
-
## Commands
|
|
113
|
-
|
|
114
|
-
| Command | Purpose |
|
|
115
|
-
|---------|---------|
|
|
116
|
-
| `selftune eval generate --list-skills` | Inventory all skills with query counts |
|
|
117
|
-
| `selftune status` | Health snapshot across all skills |
|
|
118
|
-
| `selftune eval generate --skill <name> --stats` | Per-skill aggregate telemetry |
|
|
119
|
-
| `selftune eval generate --skill <name> --max 50` | Generate eval set per skill |
|
|
120
|
-
|
|
121
|
-
## Output
|
|
122
|
-
|
|
123
|
-
Produce a structured pattern analysis report:
|
|
124
|
-
|
|
125
|
-
```markdown
|
|
126
|
-
## Cross-Skill Pattern Analysis
|
|
127
|
-
|
|
128
|
-
### Skill Inventory
|
|
129
|
-
| Skill | Sessions | Pass Rate | Status |
|
|
130
|
-
|-------|----------|-----------|--------|
|
|
131
|
-
| ... | ... | ... | ... |
|
|
132
|
-
|
|
133
|
-
### Trigger Conflicts
|
|
134
|
-
[List of conflicting trigger pairs with affected queries]
|
|
135
|
-
|
|
136
|
-
| Skill A | Skill B | Shared Triggers | Affected Queries |
|
|
137
|
-
|---------|---------|-----------------|------------------|
|
|
138
|
-
| ... | ... | ... | ... |
|
|
139
|
-
|
|
140
|
-
### Coverage Gaps
|
|
141
|
-
[Queries from all_queries_log that matched no skill]
|
|
142
|
-
|
|
143
|
-
### Misroutes
|
|
144
|
-
[Queries that triggered the wrong skill based on intent analysis]
|
|
145
|
-
|
|
146
|
-
### Systemic Issues
|
|
147
|
-
[Problems affecting multiple skills: shared infrastructure,
|
|
148
|
-
common failure patterns, evolution interference]
|
|
149
|
-
|
|
150
|
-
### Optimization Recommendations
|
|
151
|
-
1. [Highest impact change]
|
|
152
|
-
2. [Secondary optimization]
|
|
153
|
-
3. [Future consideration]
|
|
154
|
-
|
|
155
|
-
### Conflict Resolution Plan
|
|
156
|
-
[For each conflict, a specific resolution:]
|
|
157
|
-
- Skill A should own: [queries]
|
|
158
|
-
- Skill B should own: [queries]
|
|
159
|
-
- Add negative examples to: [skill]
|
|
160
|
-
```
|