selftune 0.1.4 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/diagnosis-analyst.md +156 -0
- package/.claude/agents/evolution-reviewer.md +180 -0
- package/.claude/agents/integration-guide.md +212 -0
- package/.claude/agents/pattern-analyst.md +160 -0
- package/CHANGELOG.md +46 -1
- package/README.md +105 -257
- package/apps/local-dashboard/dist/assets/geist-cyrillic-wght-normal-CHSlOQsW.woff2 +0 -0
- package/apps/local-dashboard/dist/assets/geist-latin-ext-wght-normal-DMtmJ5ZE.woff2 +0 -0
- package/apps/local-dashboard/dist/assets/geist-latin-wght-normal-Dm3htQBi.woff2 +0 -0
- package/apps/local-dashboard/dist/assets/index-C4EOTFZ2.js +15 -0
- package/apps/local-dashboard/dist/assets/index-bl-Webyd.css +1 -0
- package/apps/local-dashboard/dist/assets/vendor-react-U7zYD9Rg.js +60 -0
- package/apps/local-dashboard/dist/assets/vendor-table-B7VF2Ipl.js +26 -0
- package/apps/local-dashboard/dist/assets/vendor-ui-D7_zX_qy.js +346 -0
- package/apps/local-dashboard/dist/favicon.png +0 -0
- package/apps/local-dashboard/dist/index.html +17 -0
- package/apps/local-dashboard/dist/logo.png +0 -0
- package/apps/local-dashboard/dist/logo.svg +9 -0
- package/assets/BeforeAfter.gif +0 -0
- package/assets/FeedbackLoop.gif +0 -0
- package/assets/logo.svg +9 -0
- package/assets/skill-health-badge.svg +20 -0
- package/cli/selftune/activation-rules.ts +171 -0
- package/cli/selftune/badge/badge-data.ts +108 -0
- package/cli/selftune/badge/badge-svg.ts +212 -0
- package/cli/selftune/badge/badge.ts +99 -0
- package/cli/selftune/canonical-export.ts +183 -0
- package/cli/selftune/constants.ts +103 -1
- package/cli/selftune/contribute/bundle.ts +314 -0
- package/cli/selftune/contribute/contribute.ts +214 -0
- package/cli/selftune/contribute/sanitize.ts +162 -0
- package/cli/selftune/cron/setup.ts +266 -0
- package/cli/selftune/dashboard-contract.ts +202 -0
- package/cli/selftune/dashboard-server.ts +1049 -0
- package/cli/selftune/dashboard.ts +43 -156
- package/cli/selftune/eval/baseline.ts +248 -0
- package/cli/selftune/eval/composability-v2.ts +273 -0
- package/cli/selftune/eval/composability.ts +117 -0
- package/cli/selftune/eval/generate-unit-tests.ts +143 -0
- package/cli/selftune/eval/hooks-to-evals.ts +101 -16
- package/cli/selftune/eval/import-skillsbench.ts +221 -0
- package/cli/selftune/eval/synthetic-evals.ts +172 -0
- package/cli/selftune/eval/unit-test-cli.ts +152 -0
- package/cli/selftune/eval/unit-test.ts +196 -0
- package/cli/selftune/evolution/deploy-proposal.ts +142 -1
- package/cli/selftune/evolution/evidence.ts +26 -0
- package/cli/selftune/evolution/evolve-body.ts +586 -0
- package/cli/selftune/evolution/evolve.ts +825 -116
- package/cli/selftune/evolution/extract-patterns.ts +105 -16
- package/cli/selftune/evolution/pareto.ts +314 -0
- package/cli/selftune/evolution/propose-body.ts +171 -0
- package/cli/selftune/evolution/propose-description.ts +100 -2
- package/cli/selftune/evolution/propose-routing.ts +166 -0
- package/cli/selftune/evolution/refine-body.ts +141 -0
- package/cli/selftune/evolution/rollback.ts +21 -4
- package/cli/selftune/evolution/validate-body.ts +254 -0
- package/cli/selftune/evolution/validate-proposal.ts +257 -35
- package/cli/selftune/evolution/validate-routing.ts +177 -0
- package/cli/selftune/grading/auto-grade.ts +200 -0
- package/cli/selftune/grading/grade-session.ts +513 -42
- package/cli/selftune/grading/pre-gates.ts +104 -0
- package/cli/selftune/grading/results.ts +42 -0
- package/cli/selftune/hooks/auto-activate.ts +185 -0
- package/cli/selftune/hooks/evolution-guard.ts +165 -0
- package/cli/selftune/hooks/prompt-log.ts +172 -2
- package/cli/selftune/hooks/session-stop.ts +123 -3
- package/cli/selftune/hooks/skill-change-guard.ts +112 -0
- package/cli/selftune/hooks/skill-eval.ts +119 -3
- package/cli/selftune/index.ts +415 -48
- package/cli/selftune/ingestors/claude-replay.ts +377 -0
- package/cli/selftune/ingestors/codex-rollout.ts +345 -46
- package/cli/selftune/ingestors/codex-wrapper.ts +207 -39
- package/cli/selftune/ingestors/openclaw-ingest.ts +573 -0
- package/cli/selftune/ingestors/opencode-ingest.ts +193 -17
- package/cli/selftune/init.ts +376 -16
- package/cli/selftune/last.ts +14 -5
- package/cli/selftune/localdb/db.ts +63 -0
- package/cli/selftune/localdb/materialize.ts +428 -0
- package/cli/selftune/localdb/queries.ts +376 -0
- package/cli/selftune/localdb/schema.ts +204 -0
- package/cli/selftune/memory/writer.ts +447 -0
- package/cli/selftune/monitoring/watch.ts +90 -16
- package/cli/selftune/normalization.ts +682 -0
- package/cli/selftune/observability.ts +19 -44
- package/cli/selftune/orchestrate.ts +1073 -0
- package/cli/selftune/quickstart.ts +203 -0
- package/cli/selftune/repair/skill-usage.ts +576 -0
- package/cli/selftune/schedule.ts +561 -0
- package/cli/selftune/status.ts +59 -33
- package/cli/selftune/sync.ts +627 -0
- package/cli/selftune/types.ts +525 -5
- package/cli/selftune/utils/canonical-log.ts +45 -0
- package/cli/selftune/utils/frontmatter.ts +217 -0
- package/cli/selftune/utils/hooks.ts +41 -0
- package/cli/selftune/utils/html.ts +27 -0
- package/cli/selftune/utils/llm-call.ts +103 -19
- package/cli/selftune/utils/math.ts +10 -0
- package/cli/selftune/utils/query-filter.ts +139 -0
- package/cli/selftune/utils/skill-discovery.ts +340 -0
- package/cli/selftune/utils/skill-log.ts +68 -0
- package/cli/selftune/utils/skill-usage-confidence.ts +18 -0
- package/cli/selftune/utils/transcript.ts +307 -26
- package/cli/selftune/utils/trigger-check.ts +89 -0
- package/cli/selftune/utils/tui.ts +156 -0
- package/cli/selftune/workflows/discover.ts +254 -0
- package/cli/selftune/workflows/skill-md-writer.ts +288 -0
- package/cli/selftune/workflows/workflows.ts +188 -0
- package/package.json +28 -11
- package/packages/telemetry-contract/README.md +11 -0
- package/packages/telemetry-contract/fixtures/golden.json +87 -0
- package/packages/telemetry-contract/fixtures/golden.test.ts +42 -0
- package/packages/telemetry-contract/index.ts +1 -0
- package/packages/telemetry-contract/package.json +19 -0
- package/packages/telemetry-contract/src/index.ts +2 -0
- package/packages/telemetry-contract/src/types.ts +163 -0
- package/packages/telemetry-contract/src/validators.ts +109 -0
- package/skill/SKILL.md +180 -33
- package/skill/Workflows/AutoActivation.md +145 -0
- package/skill/Workflows/Badge.md +124 -0
- package/skill/Workflows/Baseline.md +144 -0
- package/skill/Workflows/Composability.md +107 -0
- package/skill/Workflows/Contribute.md +94 -0
- package/skill/Workflows/Cron.md +132 -0
- package/skill/Workflows/Dashboard.md +214 -0
- package/skill/Workflows/Doctor.md +63 -14
- package/skill/Workflows/Evals.md +110 -18
- package/skill/Workflows/EvolutionMemory.md +154 -0
- package/skill/Workflows/Evolve.md +181 -21
- package/skill/Workflows/EvolveBody.md +159 -0
- package/skill/Workflows/Grade.md +36 -31
- package/skill/Workflows/ImportSkillsBench.md +117 -0
- package/skill/Workflows/Ingest.md +142 -21
- package/skill/Workflows/Initialize.md +91 -23
- package/skill/Workflows/Orchestrate.md +139 -0
- package/skill/Workflows/Replay.md +91 -0
- package/skill/Workflows/Rollback.md +23 -4
- package/skill/Workflows/Schedule.md +61 -0
- package/skill/Workflows/Sync.md +88 -0
- package/skill/Workflows/UnitTest.md +150 -0
- package/skill/Workflows/Watch.md +33 -1
- package/skill/Workflows/Workflows.md +129 -0
- package/skill/assets/activation-rules-default.json +26 -0
- package/skill/assets/multi-skill-settings.json +63 -0
- package/skill/assets/single-skill-settings.json +57 -0
- package/skill/references/invocation-taxonomy.md +2 -2
- package/skill/references/logs.md +164 -2
- package/skill/references/setup-patterns.md +65 -0
- package/skill/references/version-history.md +40 -0
- package/skill/settings_snippet.json +23 -0
- package/templates/activation-rules-default.json +27 -0
- package/templates/multi-skill-settings.json +64 -0
- package/templates/single-skill-settings.json +58 -0
- package/dashboard/index.html +0 -1119
|
@@ -1,27 +1,79 @@
|
|
|
1
1
|
# selftune Ingest Workflow
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
> **Note:** Claude Code is the fully supported platform. Codex, OpenCode, and OpenClaw adapters are experimental and may have gaps.
|
|
4
|
+
|
|
5
|
+
Import sessions from agent platforms into the shared selftune log format.
|
|
6
|
+
Covers five sub-commands: `ingest claude`, `ingest codex`, `ingest opencode`,
|
|
7
|
+
`ingest openclaw`, and `ingest wrap-codex`.
|
|
6
8
|
|
|
7
9
|
## When to Use Each
|
|
8
10
|
|
|
9
11
|
| Sub-command | Platform | Mode | When |
|
|
10
12
|
|-------------|----------|------|------|
|
|
11
|
-
| `ingest
|
|
12
|
-
| `ingest
|
|
13
|
-
| `
|
|
13
|
+
| `ingest claude` | Claude Code | Batch | Backfill logs from existing Claude Code transcripts |
|
|
14
|
+
| `ingest codex` | Codex | Batch | Import existing Codex rollout logs |
|
|
15
|
+
| `ingest opencode` | OpenCode | Batch | Import existing OpenCode sessions |
|
|
16
|
+
| `ingest openclaw` | OpenClaw | Batch | Import existing OpenClaw agent sessions |
|
|
17
|
+
| `ingest wrap-codex` | Codex | Real-time | Wrap `codex exec` to capture telemetry live |
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## ingest claude
|
|
22
|
+
|
|
23
|
+
Batch ingest existing Claude Code session transcripts into the shared JSONL schema.
|
|
24
|
+
|
|
25
|
+
### Default Command
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
selftune ingest claude
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
### Options
|
|
32
|
+
|
|
33
|
+
| Flag | Description |
|
|
34
|
+
|------|-------------|
|
|
35
|
+
| `--since <date>` | Only ingest sessions modified after this date (e.g., `2026-01-01`) |
|
|
36
|
+
| `--dry-run` | Show what would be ingested without writing to logs |
|
|
37
|
+
| `--force` | Re-ingest all sessions, ignoring the marker file |
|
|
38
|
+
| `--verbose` | Show per-file progress during ingestion |
|
|
39
|
+
| `--projects-dir <path>` | Override default `~/.claude/projects/` directory |
|
|
40
|
+
|
|
41
|
+
### Source
|
|
42
|
+
|
|
43
|
+
Reads from `~/.claude/projects/<hash>/<session-id>.jsonl`. These are the
|
|
44
|
+
transcript files Claude Code automatically saves for every session.
|
|
45
|
+
|
|
46
|
+
### Output
|
|
47
|
+
|
|
48
|
+
Writes to:
|
|
49
|
+
- `~/.claude/all_queries_log.jsonl` -- extracted user queries (one per query, not just last)
|
|
50
|
+
- `~/.claude/session_telemetry_log.jsonl` -- per-session metrics with `source: "claude_code_replay"`
|
|
51
|
+
- `~/.claude/skill_usage_log.jsonl` -- skill triggers with `source: "claude_code_replay"`
|
|
52
|
+
|
|
53
|
+
### Steps
|
|
54
|
+
|
|
55
|
+
1. Run `selftune ingest claude --dry-run` to preview what would be ingested
|
|
56
|
+
2. Run `selftune ingest claude` to ingest all sessions
|
|
57
|
+
3. Run `selftune doctor` to confirm logs are healthy
|
|
58
|
+
4. Run `selftune eval generate --list-skills` to see if the ingested sessions appear
|
|
59
|
+
|
|
60
|
+
### Notes
|
|
61
|
+
|
|
62
|
+
- Idempotent: uses a marker file (`~/.claude/claude_code_ingested_sessions.json`) to track
|
|
63
|
+
which transcripts have already been ingested. Safe to run repeatedly.
|
|
64
|
+
- Extracts ALL user queries per session, not just the last one.
|
|
65
|
+
- Filters out system messages, short queries (<4 chars), and queries matching `SKIP_PREFIXES`.
|
|
14
66
|
|
|
15
67
|
---
|
|
16
68
|
|
|
17
|
-
## ingest
|
|
69
|
+
## ingest codex
|
|
18
70
|
|
|
19
71
|
Batch ingest Codex rollout logs into the shared JSONL schema.
|
|
20
72
|
|
|
21
73
|
### Default Command
|
|
22
74
|
|
|
23
75
|
```bash
|
|
24
|
-
selftune ingest
|
|
76
|
+
selftune ingest codex
|
|
25
77
|
```
|
|
26
78
|
|
|
27
79
|
### Options
|
|
@@ -42,20 +94,20 @@ Writes to:
|
|
|
42
94
|
### Steps
|
|
43
95
|
|
|
44
96
|
1. Verify `$CODEX_HOME/sessions/` directory exists and contains session files
|
|
45
|
-
2. Run `selftune ingest
|
|
97
|
+
2. Run `selftune ingest codex`
|
|
46
98
|
3. Verify entries were written by checking log file line counts
|
|
47
99
|
4. Run `selftune doctor` to confirm logs are healthy
|
|
48
100
|
|
|
49
101
|
---
|
|
50
102
|
|
|
51
|
-
## ingest
|
|
103
|
+
## ingest opencode
|
|
52
104
|
|
|
53
105
|
Ingest OpenCode sessions from the SQLite database.
|
|
54
106
|
|
|
55
107
|
### Default Command
|
|
56
108
|
|
|
57
109
|
```bash
|
|
58
|
-
selftune ingest
|
|
110
|
+
selftune ingest opencode
|
|
59
111
|
```
|
|
60
112
|
|
|
61
113
|
### Options
|
|
@@ -78,13 +130,65 @@ Writes to:
|
|
|
78
130
|
### Steps
|
|
79
131
|
|
|
80
132
|
1. Verify the OpenCode database exists at the expected path
|
|
81
|
-
2. Run `selftune ingest
|
|
133
|
+
2. Run `selftune ingest opencode`
|
|
82
134
|
3. Verify entries were written by checking log file line counts
|
|
83
135
|
4. Run `selftune doctor` to confirm logs are healthy
|
|
84
136
|
|
|
85
137
|
---
|
|
86
138
|
|
|
87
|
-
##
|
|
139
|
+
## ingest openclaw
|
|
140
|
+
|
|
141
|
+
Batch ingest OpenClaw agent session histories into the shared JSONL schema.
|
|
142
|
+
Supports multiple agents and auto-discovers session files across all agent directories.
|
|
143
|
+
|
|
144
|
+
### Default Command
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
selftune ingest openclaw
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
### Options
|
|
151
|
+
|
|
152
|
+
| Flag | Description |
|
|
153
|
+
|------|-------------|
|
|
154
|
+
| `--agents-dir <path>` | Override default `~/.openclaw/agents/` directory |
|
|
155
|
+
| `--since <date>` | Only ingest sessions modified after this date (e.g., `2026-01-01`) |
|
|
156
|
+
| `--dry-run` | Show what would be ingested without writing to logs |
|
|
157
|
+
| `--force` | Re-ingest all sessions, ignoring the marker file |
|
|
158
|
+
| `--verbose` / `-v` | Show per-session progress during ingestion |
|
|
159
|
+
|
|
160
|
+
### Source
|
|
161
|
+
|
|
162
|
+
Reads from `~/.openclaw/agents/<agentId>/sessions/*.jsonl`. Each JSONL file contains:
|
|
163
|
+
- Line 1 (session header): `{"type":"session","version":5,"id":"<uuid>","timestamp":"<iso>","cwd":"<path>"}`
|
|
164
|
+
- Line 2+ (messages): `{"role":"user|assistant|toolResult","content":[...],"timestamp":<ms>}`
|
|
165
|
+
|
|
166
|
+
### Output
|
|
167
|
+
|
|
168
|
+
Writes to:
|
|
169
|
+
- `~/.claude/all_queries_log.jsonl` -- extracted user queries
|
|
170
|
+
- `~/.claude/session_telemetry_log.jsonl` -- per-session metrics with `source: "openclaw"`
|
|
171
|
+
- `~/.claude/skill_usage_log.jsonl` -- skill triggers with `source: "openclaw"`
|
|
172
|
+
|
|
173
|
+
### Steps
|
|
174
|
+
|
|
175
|
+
1. Run `selftune ingest openclaw --dry-run` to preview what would be ingested
|
|
176
|
+
2. Run `selftune ingest openclaw` to ingest all sessions
|
|
177
|
+
3. Run `selftune doctor` to confirm logs are healthy
|
|
178
|
+
4. Run `selftune eval generate --list-skills` to see if the ingested sessions appear
|
|
179
|
+
|
|
180
|
+
### Notes
|
|
181
|
+
|
|
182
|
+
- Idempotent: uses a marker file to track which sessions have already been ingested.
|
|
183
|
+
Safe to run repeatedly. Use `--force` to re-ingest everything.
|
|
184
|
+
- Skill detection heuristic: identifies skills by checking for `SKILL.md` file reads in
|
|
185
|
+
tool calls and by matching known skill names in assistant text content.
|
|
186
|
+
- Multi-agent support: scans all agent directories under the agents root, ingesting
|
|
187
|
+
sessions from every agent found.
|
|
188
|
+
|
|
189
|
+
---
|
|
190
|
+
|
|
191
|
+
## ingest wrap-codex
|
|
88
192
|
|
|
89
193
|
Wrap `codex exec` with real-time telemetry capture. Drop-in replacement
|
|
90
194
|
that tees the JSONL stream while passing through to Codex.
|
|
@@ -92,7 +196,7 @@ that tees the JSONL stream while passing through to Codex.
|
|
|
92
196
|
### Default Command
|
|
93
197
|
|
|
94
198
|
```bash
|
|
95
|
-
selftune wrap-codex -- <your codex args>
|
|
199
|
+
selftune ingest wrap-codex -- <your codex args>
|
|
96
200
|
```
|
|
97
201
|
|
|
98
202
|
### Usage
|
|
@@ -100,7 +204,7 @@ selftune wrap-codex -- <your codex args>
|
|
|
100
204
|
Everything after `--` is passed directly to `codex exec`:
|
|
101
205
|
|
|
102
206
|
```bash
|
|
103
|
-
selftune wrap-codex -- --model o3 "Fix the failing tests"
|
|
207
|
+
selftune ingest wrap-codex -- --model o3 "Fix the failing tests"
|
|
104
208
|
```
|
|
105
209
|
|
|
106
210
|
### Output
|
|
@@ -119,23 +223,40 @@ stream for telemetry; it does not modify Codex behavior.
|
|
|
119
223
|
3. Session telemetry is captured automatically
|
|
120
224
|
4. Verify with `selftune doctor` after first use
|
|
121
225
|
|
|
226
|
+
If telemetry capture fails, check that the codex binary is accessible and that
|
|
227
|
+
the target working directory exists. Inspect the wrapper's stderr output for
|
|
228
|
+
error details — `wrap-codex` captures telemetry through the Codex wrapper, not
|
|
229
|
+
through hooks.
|
|
230
|
+
|
|
122
231
|
---
|
|
123
232
|
|
|
124
233
|
## Common Patterns
|
|
125
234
|
|
|
235
|
+
**"Backfill Claude Code sessions"**
|
|
236
|
+
> Run `selftune ingest claude`. No options needed. Reads from `~/.claude/projects/`.
|
|
237
|
+
|
|
238
|
+
**"Replay only recent Claude Code sessions"**
|
|
239
|
+
> Run `selftune ingest claude --since 2026-02-01` with an appropriate date.
|
|
240
|
+
|
|
126
241
|
**"Ingest codex logs"**
|
|
127
|
-
> Run `selftune ingest
|
|
242
|
+
> Run `selftune ingest codex`. No options needed. Reads from `$CODEX_HOME/sessions/`.
|
|
128
243
|
|
|
129
244
|
**"Import opencode sessions"**
|
|
130
|
-
> Run `selftune ingest
|
|
245
|
+
> Run `selftune ingest opencode`. Reads from the SQLite database automatically.
|
|
246
|
+
|
|
247
|
+
**"Ingest OpenClaw sessions"**
|
|
248
|
+
> Run `selftune ingest openclaw`. Reads from `~/.openclaw/agents/` automatically.
|
|
249
|
+
|
|
250
|
+
**"Import only recent OpenClaw sessions"**
|
|
251
|
+
> Run `selftune ingest openclaw --since 2026-02-01` with an appropriate date.
|
|
131
252
|
|
|
132
253
|
**"Run codex through selftune"**
|
|
133
|
-
> Use `selftune wrap-codex -- <codex args>` instead of `codex exec <args>` directly.
|
|
254
|
+
> Use `selftune ingest wrap-codex -- <codex args>` instead of `codex exec <args>` directly.
|
|
134
255
|
|
|
135
256
|
**"Batch ingest vs real-time"**
|
|
136
|
-
> Use `selftune ingest
|
|
137
|
-
> Use `selftune wrap-codex` for ongoing sessions. Both produce the same log format.
|
|
257
|
+
> Use `selftune ingest codex` or `selftune ingest opencode` for historical sessions.
|
|
258
|
+
> Use `selftune ingest wrap-codex` for ongoing sessions. Both produce the same log format.
|
|
138
259
|
|
|
139
260
|
**"How do I know it worked?"**
|
|
140
261
|
> Run `selftune doctor` after ingestion. Check that log files exist and are parseable.
|
|
141
|
-
> Run `selftune
|
|
262
|
+
> Run `selftune eval generate --list-skills` to see if the ingested sessions appear.
|
|
@@ -4,9 +4,9 @@ Bootstrap selftune for first-time use or after changing environments.
|
|
|
4
4
|
|
|
5
5
|
## When to Use
|
|
6
6
|
|
|
7
|
-
-
|
|
8
|
-
-
|
|
9
|
-
-
|
|
7
|
+
- The user asks to set up selftune, configure selftune, or initialize selftune
|
|
8
|
+
- The agent detects `~/.selftune/config.json` does not exist
|
|
9
|
+
- The user has switched agent platforms (Claude Code, Codex, OpenCode)
|
|
10
10
|
|
|
11
11
|
## Default Command
|
|
12
12
|
|
|
@@ -69,7 +69,7 @@ cat ~/.selftune/config.json 2>/dev/null
|
|
|
69
69
|
```
|
|
70
70
|
|
|
71
71
|
If the file exists and is valid JSON, selftune is already initialized.
|
|
72
|
-
Skip to Step
|
|
72
|
+
Skip to Step 8 (verify with doctor) unless the user wants to reinitialize.
|
|
73
73
|
|
|
74
74
|
### 3. Run Init
|
|
75
75
|
|
|
@@ -77,30 +77,79 @@ Skip to Step 5 (verify with doctor) unless the user wants to reinitialize.
|
|
|
77
77
|
selftune init
|
|
78
78
|
```
|
|
79
79
|
|
|
80
|
-
### 4.
|
|
80
|
+
### 4. Hooks (Claude Code)
|
|
81
81
|
|
|
82
|
-
|
|
83
|
-
`skill/settings_snippet.json` into
|
|
84
|
-
|
|
82
|
+
Hooks are **automatically installed** by `selftune init`. The init command
|
|
83
|
+
merges selftune hook entries from `skill/settings_snippet.json` into
|
|
84
|
+
`~/.claude/settings.json` without overwriting existing user hooks. If the
|
|
85
|
+
hooks are already present, they are skipped (no duplicates).
|
|
86
|
+
|
|
87
|
+
The init output will report what was installed, e.g.:
|
|
88
|
+
|
|
89
|
+
```text
|
|
90
|
+
[INFO] Installed 4 selftune hook(s) into ~/.claude/settings.json: UserPromptSubmit, PreToolUse, PostToolUse, Stop
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
**Hook reference** (for troubleshooting):
|
|
85
94
|
|
|
86
95
|
| Hook | Script | Purpose |
|
|
87
96
|
|------|--------|---------|
|
|
88
97
|
| `UserPromptSubmit` | `hooks/prompt-log.ts` | Log every user query |
|
|
98
|
+
| `UserPromptSubmit` | `hooks/auto-activate.ts` | Suggest skills before prompt processing |
|
|
99
|
+
| `PreToolUse` (Write/Edit) | `hooks/skill-change-guard.ts` | Detect uncontrolled skill edits |
|
|
100
|
+
| `PreToolUse` (Write/Edit) | `hooks/evolution-guard.ts` | Block SKILL.md edits on monitored skills |
|
|
89
101
|
| `PostToolUse` (Read) | `hooks/skill-eval.ts` | Track skill triggers |
|
|
90
102
|
| `Stop` | `hooks/session-stop.ts` | Capture session telemetry |
|
|
91
103
|
|
|
92
|
-
Derive the hook script paths from the `cli_path` field in `~/.selftune/config.json`.
|
|
93
|
-
The hooks directory is at `dirname(cli_path)/hooks/`.
|
|
94
|
-
|
|
95
104
|
**Codex agents:**
|
|
96
|
-
- Use `wrap-codex` for real-time telemetry capture (see `Workflows/Ingest.md`)
|
|
97
|
-
- Or batch-ingest existing sessions with `selftune ingest
|
|
105
|
+
- Use `selftune ingest wrap-codex` for real-time telemetry capture (see `Workflows/Ingest.md`)
|
|
106
|
+
- Or batch-ingest existing sessions with `selftune ingest codex`
|
|
98
107
|
|
|
99
108
|
**OpenCode agents:**
|
|
100
|
-
- Use `selftune ingest
|
|
109
|
+
- Use `selftune ingest opencode` to import sessions from the SQLite database
|
|
101
110
|
- See `Workflows/Ingest.md` for details
|
|
102
111
|
|
|
103
|
-
### 5.
|
|
112
|
+
### 5. Initialize Memory Directory
|
|
113
|
+
|
|
114
|
+
Create the memory directory if it does not exist:
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
mkdir -p ~/.selftune/memory
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
The memory system stores three files at `~/.selftune/memory/`:
|
|
121
|
+
- `context.md` -- active evolution state and session context
|
|
122
|
+
- `decisions.md` -- evolution decisions and rollback history
|
|
123
|
+
- `plan.md` -- current priorities and evolution strategy
|
|
124
|
+
|
|
125
|
+
These files are created automatically by the memory writer during evolve,
|
|
126
|
+
watch, and rollback workflows. The directory just needs to exist.
|
|
127
|
+
|
|
128
|
+
### 6. Set Up Activation Rules
|
|
129
|
+
|
|
130
|
+
`selftune init` copies the default activation rules template to
|
|
131
|
+
`~/.selftune/activation-rules.json` automatically. If the file is missing,
|
|
132
|
+
run `selftune init --force` to regenerate it.
|
|
133
|
+
|
|
134
|
+
The activation rules file configures auto-activation behavior -- which skills
|
|
135
|
+
get suggested and under what conditions. Edit `~/.selftune/activation-rules.json`
|
|
136
|
+
to customize thresholds and skill mappings for your project.
|
|
137
|
+
|
|
138
|
+
### 7. Verify Agent Availability
|
|
139
|
+
|
|
140
|
+
`selftune init` installs the specialized agent files to `~/.claude/agents/`
|
|
141
|
+
automatically. Verify they are present:
|
|
142
|
+
|
|
143
|
+
```bash
|
|
144
|
+
ls ~/.claude/agents/
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Expected agents: `diagnosis-analyst.md`, `pattern-analyst.md`,
|
|
148
|
+
`evolution-reviewer.md`, `integration-guide.md`. These are used by evolve
|
|
149
|
+
and doctor workflows for deeper analysis. If missing, run `selftune init --force`
|
|
150
|
+
to reinstall them.
|
|
151
|
+
|
|
152
|
+
### 8. Verify with Doctor
|
|
104
153
|
|
|
105
154
|
```bash
|
|
106
155
|
selftune doctor
|
|
@@ -109,15 +158,34 @@ selftune doctor
|
|
|
109
158
|
Parse the JSON output. All checks should pass. If any fail, address the
|
|
110
159
|
reported issues before proceeding.
|
|
111
160
|
|
|
161
|
+
## Integration Guide
|
|
162
|
+
|
|
163
|
+
For project-type-specific setup (single-skill, multi-skill, monorepo, Codex,
|
|
164
|
+
OpenCode, mixed agents), see [docs/integration-guide.md](../../docs/integration-guide.md).
|
|
165
|
+
|
|
166
|
+
Templates for each project type are in the `templates/` directory:
|
|
167
|
+
- `templates/single-skill-settings.json` — hooks for single-skill projects
|
|
168
|
+
- `templates/multi-skill-settings.json` — hooks for multi-skill projects with activation rules
|
|
169
|
+
- `templates/activation-rules-default.json` — default auto-activation rule configuration
|
|
170
|
+
|
|
171
|
+
## Subagent Escalation
|
|
172
|
+
|
|
173
|
+
For complex project structures (monorepos, multi-skill repos, mixed agent
|
|
174
|
+
platforms), spawn the `integration-guide` agent as a subagent for guided
|
|
175
|
+
setup. This agent handles project-type detection, per-package configuration,
|
|
176
|
+
and verification steps that go beyond what the basic init workflow covers.
|
|
177
|
+
|
|
112
178
|
## Common Patterns
|
|
113
179
|
|
|
114
|
-
**
|
|
115
|
-
>
|
|
116
|
-
> install
|
|
180
|
+
**User asks to set up or initialize selftune**
|
|
181
|
+
> Run `which selftune` to check installation. If missing, install with
|
|
182
|
+
> `npm install -g selftune`. Run `selftune init`, then verify with
|
|
183
|
+
> `selftune doctor`. Report results to the user.
|
|
117
184
|
|
|
118
|
-
**
|
|
119
|
-
> Run `selftune doctor` to check hook installation.
|
|
120
|
-
>
|
|
185
|
+
**Hooks not capturing data**
|
|
186
|
+
> Run `selftune doctor` to check hook installation. Parse the JSON output
|
|
187
|
+
> for failed hook checks. If paths are wrong, update
|
|
188
|
+
> `~/.claude/settings.json` to point to actual files.
|
|
121
189
|
|
|
122
|
-
**
|
|
123
|
-
> Run `selftune init --force` to reinitialize.
|
|
190
|
+
**Config exists but appears stale**
|
|
191
|
+
> Run `selftune init --force` to reinitialize. Verify with `selftune doctor`.
|
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
# selftune Orchestrate Workflow
|
|
2
|
+
|
|
3
|
+
Run the autonomy-first selftune loop in one command.
|
|
4
|
+
|
|
5
|
+
`selftune orchestrate` is the primary closed-loop entrypoint. It runs
|
|
6
|
+
source-truth sync, computes current skill health, selects candidates,
|
|
7
|
+
deploys validated low-risk description changes autonomously, and watches
|
|
8
|
+
recent changes with auto-rollback enabled.
|
|
9
|
+
|
|
10
|
+
## When to Use
|
|
11
|
+
|
|
12
|
+
- You want the full autonomous loop, not isolated subcommands
|
|
13
|
+
- You want to improve skills without manually chaining `sync`, `status`, `evolve`, and `watch`
|
|
14
|
+
- You want a dry-run of what selftune would change next
|
|
15
|
+
- You want a stricter review policy for a single run
|
|
16
|
+
|
|
17
|
+
## Default Command
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
selftune orchestrate
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Flags
|
|
24
|
+
|
|
25
|
+
| Flag | Description | Default |
|
|
26
|
+
|------|-------------|---------|
|
|
27
|
+
| `--dry-run` | Plan and validate without deploying changes | Off |
|
|
28
|
+
| `--review-required` | Keep validated changes in review mode instead of deploying | Off |
|
|
29
|
+
| `--skill <name>` | Limit the loop to one skill | All skills |
|
|
30
|
+
| `--max-skills <n>` | Cap how many candidates are processed in one run | `3` |
|
|
31
|
+
| `--recent-window <hours>` | Window for post-deploy watch/rollback checks | `24` |
|
|
32
|
+
| `--sync-force` | Force a full source replay before candidate selection | Off |
|
|
33
|
+
|
|
34
|
+
## Default Behavior
|
|
35
|
+
|
|
36
|
+
- Sync source-truth telemetry first
|
|
37
|
+
- Prioritize critical/warning/ungraded skills with real missed-query signal
|
|
38
|
+
- Deploy validated low-risk description changes automatically
|
|
39
|
+
- Watch recent deployments and roll back regressions automatically
|
|
40
|
+
|
|
41
|
+
Use `--review-required` only when you want a stricter policy for a specific run.
|
|
42
|
+
|
|
43
|
+
## Common Patterns
|
|
44
|
+
|
|
45
|
+
**User asks to improve skills or run the full loop**
|
|
46
|
+
> Run `selftune orchestrate`. Parse the JSON output from stdout and the
|
|
47
|
+
> phased report from stderr. Report the summary to the user.
|
|
48
|
+
|
|
49
|
+
**User wants to preview changes before deploying**
|
|
50
|
+
> Run `selftune orchestrate --dry-run`. Report the planned actions without
|
|
51
|
+
> making any changes.
|
|
52
|
+
|
|
53
|
+
**User wants to focus on a single skill**
|
|
54
|
+
> Run `selftune orchestrate --skill <name>`. This limits the loop to the
|
|
55
|
+
> specified skill only.
|
|
56
|
+
|
|
57
|
+
**User wants manual review before deployment**
|
|
58
|
+
> Run `selftune orchestrate --review-required`. Validated changes stay in
|
|
59
|
+
> review mode instead of auto-deploying.
|
|
60
|
+
|
|
61
|
+
**Agent needs fresh source data before orchestrating**
|
|
62
|
+
> Run `selftune orchestrate --sync-force`. This forces a full source replay
|
|
63
|
+
> before candidate selection.
|
|
64
|
+
|
|
65
|
+
## Output
|
|
66
|
+
|
|
67
|
+
### Human-readable report (stderr)
|
|
68
|
+
|
|
69
|
+
A phased decision report printed to stderr so you can see exactly what happened and why:
|
|
70
|
+
|
|
71
|
+
1. **Phase 1: Sync** — which sources were scanned, how many records synced, repair counts
|
|
72
|
+
2. **Phase 2: Status** — skill count, system health, breakdown by status category
|
|
73
|
+
3. **Phase 3: Skill Decisions** — each skill with its action (EVOLVE / WATCH / SKIP) and reason
|
|
74
|
+
4. **Phase 4: Evolution Results** — validation pass-rate changes (before → after), deployment status
|
|
75
|
+
5. **Phase 5: Watch** — post-deploy monitoring with alert and rollback indicators
|
|
76
|
+
6. **Summary** — evaluated/deployed/watched/skipped counts and elapsed time
|
|
77
|
+
|
|
78
|
+
A mode banner at the top shows DRY RUN, REVIEW, or AUTONOMOUS with rerun hints when applicable.
|
|
79
|
+
|
|
80
|
+
### JSON output (stdout)
|
|
81
|
+
|
|
82
|
+
Machine-readable JSON with the summary fields plus a `decisions` array containing per-skill:
|
|
83
|
+
|
|
84
|
+
- `skill`, `action`, `reason`
|
|
85
|
+
- `deployed`, `evolveReason`, `validation` (before/after pass rates, improved flag) — when evolved
|
|
86
|
+
- `alert`, `rolledBack`, `passRate`, `recommendation` — when watched
|
|
87
|
+
|
|
88
|
+
This is the recommended runtime for recurring autonomous scheduling.
|
|
89
|
+
|
|
90
|
+
## Two Execution Contexts
|
|
91
|
+
|
|
92
|
+
`selftune orchestrate` runs in two contexts with different callers:
|
|
93
|
+
|
|
94
|
+
| Context | Caller | Token cost | When |
|
|
95
|
+
|---------|--------|------------|------|
|
|
96
|
+
| **Interactive** | Agent (user says "improve my skills") | Uses agent subscription | On demand |
|
|
97
|
+
| **Automated (cron)** | OS scheduler (cron/launchd/systemd) | No agent session; LLM cost only if evolution triggers | Every 6 hours |
|
|
98
|
+
| **Automated (loop)** | `selftune orchestrate --loop` | No agent session; LLM cost only if evolution triggers | Configurable interval |
|
|
99
|
+
|
|
100
|
+
In automated mode, the OS calls the CLI binary directly. No agent session
|
|
101
|
+
is created. LLM calls only happen during the evolution step (proposing and
|
|
102
|
+
validating description changes), which uses the configured model tier.
|
|
103
|
+
The orchestrate logic itself (sync, status, candidate selection) is pure
|
|
104
|
+
data processing with zero token cost.
|
|
105
|
+
|
|
106
|
+
**Cron mode:** Install OS-level scheduling with `selftune cron setup`.
|
|
107
|
+
Runs as separate invocations on a schedule (default: every 6 hours).
|
|
108
|
+
|
|
109
|
+
**Loop mode:** Run `selftune orchestrate --loop` for a long-running process
|
|
110
|
+
that cycles continuously. Use `--loop-interval <seconds>` to set the pause
|
|
111
|
+
between cycles (default: 3600s / 1 hour, minimum: 60s). Stop with Ctrl+C
|
|
112
|
+
or SIGTERM — the current cycle finishes before exit.
|
|
113
|
+
|
|
114
|
+
### Signal-Reactive Trigger
|
|
115
|
+
|
|
116
|
+
When improvement signals are detected during a session (corrections, explicit
|
|
117
|
+
requests, manual invocations), the `session-stop` hook automatically spawns a
|
|
118
|
+
focused `selftune orchestrate --max-skills 2` run in the background. This
|
|
119
|
+
reactive path complements the scheduled cron/loop modes by responding to signals
|
|
120
|
+
immediately after the session that produced them.
|
|
121
|
+
|
|
122
|
+
Guard rails:
|
|
123
|
+
- Only spawns if unconsumed signals exist in `improvement_signals.jsonl`
|
|
124
|
+
- Respects the orchestrate lock file — skips if another run started within 30 minutes
|
|
125
|
+
- Fire-and-forget: the hook exits immediately, orchestrate runs independently
|
|
126
|
+
- Silent failure: any error is swallowed so the hook never blocks Claude
|
|
127
|
+
|
|
128
|
+
### Internal Workflow Chain (Autonomous Mode)
|
|
129
|
+
|
|
130
|
+
In autonomous mode, orchestrate calls sub-workflows in this fixed order:
|
|
131
|
+
|
|
132
|
+
1. **Sync** — refresh source-truth telemetry across all supported agents (`selftune sync`)
|
|
133
|
+
2. **Status** — compute skill health using existing grade results (reads `grading.json` outputs from previous sessions)
|
|
134
|
+
3. **Evolve** — run evolution on selected candidates (pre-flight is skipped, cheap-loop mode enabled, defaults used)
|
|
135
|
+
4. **Watch** — monitor recently evolved skills (auto-rollback enabled by default, `--recent-window` hours lookback)
|
|
136
|
+
|
|
137
|
+
All sub-workflows run with defaults and no user interaction. The safety
|
|
138
|
+
model relies on regression thresholds, automatic rollback, and SKILL.md
|
|
139
|
+
backups rather than human confirmation.
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# selftune Ingest (Claude) Workflow
|
|
2
|
+
|
|
3
|
+
> **Note:** This workflow documents `selftune ingest claude`. The command was
|
|
4
|
+
> renamed from `selftune replay` to `selftune ingest claude`. This file is
|
|
5
|
+
> kept as `Replay.md` for routing compatibility.
|
|
6
|
+
|
|
7
|
+
Backfill the shared JSONL logs from existing Claude Code conversation
|
|
8
|
+
transcripts. Useful for bootstrapping selftune with historical session data.
|
|
9
|
+
|
|
10
|
+
## When to Use
|
|
11
|
+
|
|
12
|
+
- The user has a new selftune installation with months of Claude Code history
|
|
13
|
+
- The user re-initialized logs and wants to recover data
|
|
14
|
+
- The agent needs to populate eval data without waiting for new sessions
|
|
15
|
+
|
|
16
|
+
## Key Difference from Hooks
|
|
17
|
+
|
|
18
|
+
Real-time hooks capture only the **last** user query per session. Ingest
|
|
19
|
+
extracts **all** user queries, writing one `QueryLogRecord` per message.
|
|
20
|
+
This produces much richer eval data from historical sessions.
|
|
21
|
+
|
|
22
|
+
## Default Command
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
selftune ingest claude
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Options
|
|
29
|
+
|
|
30
|
+
| Flag | Description |
|
|
31
|
+
|------|-------------|
|
|
32
|
+
| `--since <date>` | Only include transcripts modified after this date |
|
|
33
|
+
| `--dry-run` | Preview what would be ingested without writing |
|
|
34
|
+
| `--force` | Re-ingest all transcripts (ignore marker file) |
|
|
35
|
+
| `--verbose` | Show detailed progress per file |
|
|
36
|
+
| `--projects-dir <path>` | Override default `~/.claude/projects/` path |
|
|
37
|
+
|
|
38
|
+
## Source
|
|
39
|
+
|
|
40
|
+
Reads Claude Code transcripts from `~/.claude/projects/<hash>/<session>.jsonl`.
|
|
41
|
+
Each transcript is a JSONL file containing user and assistant messages.
|
|
42
|
+
|
|
43
|
+
## Output
|
|
44
|
+
|
|
45
|
+
Writes to:
|
|
46
|
+
- `~/.claude/all_queries_log.jsonl` -- one record per user query (all messages, not just last)
|
|
47
|
+
- `~/.claude/session_telemetry_log.jsonl` -- per-session metrics with `source: "claude_code_replay"`
|
|
48
|
+
- `~/.claude/skill_usage_log.jsonl` -- skill triggers detected in transcripts
|
|
49
|
+
|
|
50
|
+
## Idempotency
|
|
51
|
+
|
|
52
|
+
Uses a marker file at `~/.claude/claude_code_ingested_sessions.json` to track
|
|
53
|
+
which transcripts have already been ingested. Use `--force` to re-ingest all.
|
|
54
|
+
|
|
55
|
+
## Steps
|
|
56
|
+
|
|
57
|
+
### 1. Preview Ingestion
|
|
58
|
+
|
|
59
|
+
Run `selftune ingest claude --dry-run`. Parse the output to check how many
|
|
60
|
+
transcripts would be ingested. Report the count to the user.
|
|
61
|
+
|
|
62
|
+
### 2. Run Ingestion
|
|
63
|
+
|
|
64
|
+
Run `selftune ingest claude`. Parse the output for ingested session counts
|
|
65
|
+
and any errors.
|
|
66
|
+
|
|
67
|
+
### 3. Verify Results
|
|
68
|
+
|
|
69
|
+
Run `selftune doctor` to verify logs are healthy. Run
|
|
70
|
+
`selftune eval generate --list-skills` to confirm ingested sessions appear.
|
|
71
|
+
|
|
72
|
+
### 4. Report Results
|
|
73
|
+
|
|
74
|
+
Report the number of sessions ingested and any skills discovered to the user.
|
|
75
|
+
|
|
76
|
+
## Common Patterns
|
|
77
|
+
|
|
78
|
+
**User wants to backfill logs from Claude Code history**
|
|
79
|
+
> Run `selftune ingest claude`. No options needed for a full backfill.
|
|
80
|
+
> Parse the output and report ingested session counts.
|
|
81
|
+
|
|
82
|
+
**User wants to ingest only recent sessions**
|
|
83
|
+
> Run `selftune ingest claude --since <date>` with the user's specified date.
|
|
84
|
+
|
|
85
|
+
**User wants to re-ingest everything from scratch**
|
|
86
|
+
> Run `selftune ingest claude --force`. This ignores the marker file and
|
|
87
|
+
> rescans all transcripts.
|
|
88
|
+
|
|
89
|
+
**Agent needs to verify ingestion succeeded**
|
|
90
|
+
> Run `selftune doctor` after ingestion. Parse the JSON output to check
|
|
91
|
+
> that log file entry counts increased.
|