selftune 0.2.0 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/diagnosis-analyst.md +20 -10
- package/.claude/agents/evolution-reviewer.md +14 -1
- package/.claude/agents/integration-guide.md +18 -6
- package/.claude/agents/pattern-analyst.md +18 -5
- package/CHANGELOG.md +12 -4
- package/README.md +43 -35
- package/apps/local-dashboard/dist/assets/geist-cyrillic-wght-normal-CHSlOQsW.woff2 +0 -0
- package/apps/local-dashboard/dist/assets/geist-latin-ext-wght-normal-DMtmJ5ZE.woff2 +0 -0
- package/apps/local-dashboard/dist/assets/geist-latin-wght-normal-Dm3htQBi.woff2 +0 -0
- package/apps/local-dashboard/dist/assets/index-C4EOTFZ2.js +15 -0
- package/apps/local-dashboard/dist/assets/index-bl-Webyd.css +1 -0
- package/apps/local-dashboard/dist/assets/vendor-react-U7zYD9Rg.js +60 -0
- package/apps/local-dashboard/dist/assets/vendor-table-B7VF2Ipl.js +26 -0
- package/apps/local-dashboard/dist/assets/vendor-ui-D7_zX_qy.js +346 -0
- package/apps/local-dashboard/dist/favicon.png +0 -0
- package/apps/local-dashboard/dist/index.html +17 -0
- package/apps/local-dashboard/dist/logo.png +0 -0
- package/apps/local-dashboard/dist/logo.svg +9 -0
- package/cli/selftune/badge/badge-data.ts +1 -1
- package/cli/selftune/badge/badge.ts +4 -8
- package/cli/selftune/canonical-export.ts +183 -0
- package/cli/selftune/constants.ts +28 -0
- package/cli/selftune/contribute/contribute.ts +1 -1
- package/cli/selftune/cron/setup.ts +17 -17
- package/cli/selftune/dashboard-contract.ts +202 -0
- package/cli/selftune/dashboard-server.ts +653 -186
- package/cli/selftune/dashboard.ts +41 -176
- package/cli/selftune/eval/baseline.ts +5 -4
- package/cli/selftune/eval/composability-v2.ts +273 -0
- package/cli/selftune/eval/hooks-to-evals.ts +34 -15
- package/cli/selftune/eval/unit-test-cli.ts +1 -1
- package/cli/selftune/evolution/evidence.ts +26 -0
- package/cli/selftune/evolution/evolve-body.ts +105 -11
- package/cli/selftune/evolution/evolve.ts +371 -25
- package/cli/selftune/evolution/extract-patterns.ts +87 -29
- package/cli/selftune/evolution/rollback.ts +2 -2
- package/cli/selftune/grading/auto-grade.ts +200 -0
- package/cli/selftune/grading/grade-session.ts +448 -97
- package/cli/selftune/grading/results.ts +42 -0
- package/cli/selftune/hooks/prompt-log.ts +172 -2
- package/cli/selftune/hooks/session-stop.ts +123 -3
- package/cli/selftune/hooks/skill-eval.ts +119 -3
- package/cli/selftune/index.ts +395 -116
- package/cli/selftune/ingestors/claude-replay.ts +140 -114
- package/cli/selftune/ingestors/codex-rollout.ts +345 -46
- package/cli/selftune/ingestors/codex-wrapper.ts +207 -39
- package/cli/selftune/ingestors/openclaw-ingest.ts +141 -8
- package/cli/selftune/ingestors/opencode-ingest.ts +193 -17
- package/cli/selftune/init.ts +227 -14
- package/cli/selftune/last.ts +14 -5
- package/cli/selftune/localdb/db.ts +63 -0
- package/cli/selftune/localdb/materialize.ts +428 -0
- package/cli/selftune/localdb/queries.ts +376 -0
- package/cli/selftune/localdb/schema.ts +204 -0
- package/cli/selftune/monitoring/watch.ts +66 -15
- package/cli/selftune/normalization.ts +682 -0
- package/cli/selftune/observability.ts +19 -44
- package/cli/selftune/orchestrate.ts +1073 -0
- package/cli/selftune/quickstart.ts +203 -0
- package/cli/selftune/repair/skill-usage.ts +576 -0
- package/cli/selftune/schedule.ts +561 -0
- package/cli/selftune/status.ts +48 -26
- package/cli/selftune/sync.ts +627 -0
- package/cli/selftune/types.ts +148 -0
- package/cli/selftune/utils/canonical-log.ts +45 -0
- package/cli/selftune/utils/hooks.ts +41 -0
- package/cli/selftune/utils/html.ts +27 -0
- package/cli/selftune/utils/llm-call.ts +78 -20
- package/cli/selftune/utils/math.ts +10 -0
- package/cli/selftune/utils/query-filter.ts +139 -0
- package/cli/selftune/utils/skill-discovery.ts +340 -0
- package/cli/selftune/utils/skill-log.ts +68 -0
- package/cli/selftune/utils/skill-usage-confidence.ts +18 -0
- package/cli/selftune/utils/transcript.ts +272 -26
- package/cli/selftune/workflows/discover.ts +254 -0
- package/cli/selftune/workflows/skill-md-writer.ts +288 -0
- package/cli/selftune/workflows/workflows.ts +188 -0
- package/package.json +21 -8
- package/packages/telemetry-contract/README.md +11 -0
- package/packages/telemetry-contract/fixtures/golden.json +87 -0
- package/packages/telemetry-contract/fixtures/golden.test.ts +42 -0
- package/packages/telemetry-contract/index.ts +1 -0
- package/packages/telemetry-contract/package.json +19 -0
- package/packages/telemetry-contract/src/index.ts +2 -0
- package/packages/telemetry-contract/src/types.ts +163 -0
- package/packages/telemetry-contract/src/validators.ts +109 -0
- package/skill/SKILL.md +84 -53
- package/skill/Workflows/AutoActivation.md +17 -16
- package/skill/Workflows/Badge.md +6 -0
- package/skill/Workflows/Baseline.md +46 -23
- package/skill/Workflows/Composability.md +12 -5
- package/skill/Workflows/Contribute.md +17 -14
- package/skill/Workflows/Cron.md +56 -79
- package/skill/Workflows/Dashboard.md +45 -34
- package/skill/Workflows/Doctor.md +30 -17
- package/skill/Workflows/Evals.md +64 -40
- package/skill/Workflows/EvolutionMemory.md +2 -0
- package/skill/Workflows/Evolve.md +102 -47
- package/skill/Workflows/EvolveBody.md +6 -6
- package/skill/Workflows/Grade.md +36 -31
- package/skill/Workflows/ImportSkillsBench.md +11 -5
- package/skill/Workflows/Ingest.md +43 -36
- package/skill/Workflows/Initialize.md +44 -30
- package/skill/Workflows/Orchestrate.md +139 -0
- package/skill/Workflows/Replay.md +39 -18
- package/skill/Workflows/Rollback.md +3 -3
- package/skill/Workflows/Schedule.md +61 -0
- package/skill/Workflows/Sync.md +88 -0
- package/skill/Workflows/UnitTest.md +34 -22
- package/skill/Workflows/Watch.md +14 -4
- package/skill/Workflows/Workflows.md +129 -0
- package/skill/assets/activation-rules-default.json +26 -0
- package/skill/assets/multi-skill-settings.json +63 -0
- package/skill/assets/single-skill-settings.json +57 -0
- package/skill/references/invocation-taxonomy.md +2 -2
- package/skill/references/logs.md +164 -2
- package/skill/references/setup-patterns.md +65 -0
- package/skill/references/version-history.md +40 -0
- package/skill/settings_snippet.json +1 -1
- package/templates/multi-skill-settings.json +7 -7
- package/templates/single-skill-settings.json +6 -6
- package/dashboard/index.html +0 -1680
|
@@ -1,29 +1,31 @@
|
|
|
1
1
|
# selftune Ingest Workflow
|
|
2
2
|
|
|
3
|
+
> **Note:** Claude Code is the fully supported platform. Codex, OpenCode, and OpenClaw adapters are experimental and may have gaps.
|
|
4
|
+
|
|
3
5
|
Import sessions from agent platforms into the shared selftune log format.
|
|
4
|
-
Covers five sub-commands: `
|
|
5
|
-
`ingest
|
|
6
|
+
Covers five sub-commands: `ingest claude`, `ingest codex`, `ingest opencode`,
|
|
7
|
+
`ingest openclaw`, and `ingest wrap-codex`.
|
|
6
8
|
|
|
7
9
|
## When to Use Each
|
|
8
10
|
|
|
9
11
|
| Sub-command | Platform | Mode | When |
|
|
10
12
|
|-------------|----------|------|------|
|
|
11
|
-
| `
|
|
12
|
-
| `ingest
|
|
13
|
-
| `ingest
|
|
14
|
-
| `ingest
|
|
15
|
-
| `wrap-codex` | Codex | Real-time | Wrap `codex exec` to capture telemetry live |
|
|
13
|
+
| `ingest claude` | Claude Code | Batch | Backfill logs from existing Claude Code transcripts |
|
|
14
|
+
| `ingest codex` | Codex | Batch | Import existing Codex rollout logs |
|
|
15
|
+
| `ingest opencode` | OpenCode | Batch | Import existing OpenCode sessions |
|
|
16
|
+
| `ingest openclaw` | OpenClaw | Batch | Import existing OpenClaw agent sessions |
|
|
17
|
+
| `ingest wrap-codex` | Codex | Real-time | Wrap `codex exec` to capture telemetry live |
|
|
16
18
|
|
|
17
19
|
---
|
|
18
20
|
|
|
19
|
-
##
|
|
21
|
+
## ingest claude
|
|
20
22
|
|
|
21
23
|
Batch ingest existing Claude Code session transcripts into the shared JSONL schema.
|
|
22
24
|
|
|
23
25
|
### Default Command
|
|
24
26
|
|
|
25
27
|
```bash
|
|
26
|
-
selftune
|
|
28
|
+
selftune ingest claude
|
|
27
29
|
```
|
|
28
30
|
|
|
29
31
|
### Options
|
|
@@ -50,10 +52,10 @@ Writes to:
|
|
|
50
52
|
|
|
51
53
|
### Steps
|
|
52
54
|
|
|
53
|
-
1. Run `selftune
|
|
54
|
-
2. Run `selftune
|
|
55
|
+
1. Run `selftune ingest claude --dry-run` to preview what would be ingested
|
|
56
|
+
2. Run `selftune ingest claude` to ingest all sessions
|
|
55
57
|
3. Run `selftune doctor` to confirm logs are healthy
|
|
56
|
-
4. Run `selftune
|
|
58
|
+
4. Run `selftune eval generate --list-skills` to see if the ingested sessions appear
|
|
57
59
|
|
|
58
60
|
### Notes
|
|
59
61
|
|
|
@@ -64,14 +66,14 @@ Writes to:
|
|
|
64
66
|
|
|
65
67
|
---
|
|
66
68
|
|
|
67
|
-
## ingest
|
|
69
|
+
## ingest codex
|
|
68
70
|
|
|
69
71
|
Batch ingest Codex rollout logs into the shared JSONL schema.
|
|
70
72
|
|
|
71
73
|
### Default Command
|
|
72
74
|
|
|
73
75
|
```bash
|
|
74
|
-
selftune ingest
|
|
76
|
+
selftune ingest codex
|
|
75
77
|
```
|
|
76
78
|
|
|
77
79
|
### Options
|
|
@@ -92,20 +94,20 @@ Writes to:
|
|
|
92
94
|
### Steps
|
|
93
95
|
|
|
94
96
|
1. Verify `$CODEX_HOME/sessions/` directory exists and contains session files
|
|
95
|
-
2. Run `selftune ingest
|
|
97
|
+
2. Run `selftune ingest codex`
|
|
96
98
|
3. Verify entries were written by checking log file line counts
|
|
97
99
|
4. Run `selftune doctor` to confirm logs are healthy
|
|
98
100
|
|
|
99
101
|
---
|
|
100
102
|
|
|
101
|
-
## ingest
|
|
103
|
+
## ingest opencode
|
|
102
104
|
|
|
103
105
|
Ingest OpenCode sessions from the SQLite database.
|
|
104
106
|
|
|
105
107
|
### Default Command
|
|
106
108
|
|
|
107
109
|
```bash
|
|
108
|
-
selftune ingest
|
|
110
|
+
selftune ingest opencode
|
|
109
111
|
```
|
|
110
112
|
|
|
111
113
|
### Options
|
|
@@ -128,13 +130,13 @@ Writes to:
|
|
|
128
130
|
### Steps
|
|
129
131
|
|
|
130
132
|
1. Verify the OpenCode database exists at the expected path
|
|
131
|
-
2. Run `selftune ingest
|
|
133
|
+
2. Run `selftune ingest opencode`
|
|
132
134
|
3. Verify entries were written by checking log file line counts
|
|
133
135
|
4. Run `selftune doctor` to confirm logs are healthy
|
|
134
136
|
|
|
135
137
|
---
|
|
136
138
|
|
|
137
|
-
## ingest
|
|
139
|
+
## ingest openclaw
|
|
138
140
|
|
|
139
141
|
Batch ingest OpenClaw agent session histories into the shared JSONL schema.
|
|
140
142
|
Supports multiple agents and auto-discovers session files across all agent directories.
|
|
@@ -142,7 +144,7 @@ Supports multiple agents and auto-discovers session files across all agent direc
|
|
|
142
144
|
### Default Command
|
|
143
145
|
|
|
144
146
|
```bash
|
|
145
|
-
selftune ingest
|
|
147
|
+
selftune ingest openclaw
|
|
146
148
|
```
|
|
147
149
|
|
|
148
150
|
### Options
|
|
@@ -170,10 +172,10 @@ Writes to:
|
|
|
170
172
|
|
|
171
173
|
### Steps
|
|
172
174
|
|
|
173
|
-
1. Run `selftune ingest
|
|
174
|
-
2. Run `selftune ingest
|
|
175
|
+
1. Run `selftune ingest openclaw --dry-run` to preview what would be ingested
|
|
176
|
+
2. Run `selftune ingest openclaw` to ingest all sessions
|
|
175
177
|
3. Run `selftune doctor` to confirm logs are healthy
|
|
176
|
-
4. Run `selftune
|
|
178
|
+
4. Run `selftune eval generate --list-skills` to see if the ingested sessions appear
|
|
177
179
|
|
|
178
180
|
### Notes
|
|
179
181
|
|
|
@@ -186,7 +188,7 @@ Writes to:
|
|
|
186
188
|
|
|
187
189
|
---
|
|
188
190
|
|
|
189
|
-
## wrap-codex
|
|
191
|
+
## ingest wrap-codex
|
|
190
192
|
|
|
191
193
|
Wrap `codex exec` with real-time telemetry capture. Drop-in replacement
|
|
192
194
|
that tees the JSONL stream while passing through to Codex.
|
|
@@ -194,7 +196,7 @@ that tees the JSONL stream while passing through to Codex.
|
|
|
194
196
|
### Default Command
|
|
195
197
|
|
|
196
198
|
```bash
|
|
197
|
-
selftune wrap-codex -- <your codex args>
|
|
199
|
+
selftune ingest wrap-codex -- <your codex args>
|
|
198
200
|
```
|
|
199
201
|
|
|
200
202
|
### Usage
|
|
@@ -202,7 +204,7 @@ selftune wrap-codex -- <your codex args>
|
|
|
202
204
|
Everything after `--` is passed directly to `codex exec`:
|
|
203
205
|
|
|
204
206
|
```bash
|
|
205
|
-
selftune wrap-codex -- --model o3 "Fix the failing tests"
|
|
207
|
+
selftune ingest wrap-codex -- --model o3 "Fix the failing tests"
|
|
206
208
|
```
|
|
207
209
|
|
|
208
210
|
### Output
|
|
@@ -221,35 +223,40 @@ stream for telemetry; it does not modify Codex behavior.
|
|
|
221
223
|
3. Session telemetry is captured automatically
|
|
222
224
|
4. Verify with `selftune doctor` after first use
|
|
223
225
|
|
|
226
|
+
If telemetry capture fails, check that the codex binary is accessible and that
|
|
227
|
+
the target working directory exists. Inspect the wrapper's stderr output for
|
|
228
|
+
error details — `wrap-codex` captures telemetry through the Codex wrapper, not
|
|
229
|
+
through hooks.
|
|
230
|
+
|
|
224
231
|
---
|
|
225
232
|
|
|
226
233
|
## Common Patterns
|
|
227
234
|
|
|
228
235
|
**"Backfill Claude Code sessions"**
|
|
229
|
-
> Run `selftune
|
|
236
|
+
> Run `selftune ingest claude`. No options needed. Reads from `~/.claude/projects/`.
|
|
230
237
|
|
|
231
238
|
**"Replay only recent Claude Code sessions"**
|
|
232
|
-
> Run `selftune
|
|
239
|
+
> Run `selftune ingest claude --since 2026-02-01` with an appropriate date.
|
|
233
240
|
|
|
234
241
|
**"Ingest codex logs"**
|
|
235
|
-
> Run `selftune ingest
|
|
242
|
+
> Run `selftune ingest codex`. No options needed. Reads from `$CODEX_HOME/sessions/`.
|
|
236
243
|
|
|
237
244
|
**"Import opencode sessions"**
|
|
238
|
-
> Run `selftune ingest
|
|
245
|
+
> Run `selftune ingest opencode`. Reads from the SQLite database automatically.
|
|
239
246
|
|
|
240
247
|
**"Ingest OpenClaw sessions"**
|
|
241
|
-
> Run `selftune ingest
|
|
248
|
+
> Run `selftune ingest openclaw`. Reads from `~/.openclaw/agents/` automatically.
|
|
242
249
|
|
|
243
250
|
**"Import only recent OpenClaw sessions"**
|
|
244
|
-
> Run `selftune ingest
|
|
251
|
+
> Run `selftune ingest openclaw --since 2026-02-01` with an appropriate date.
|
|
245
252
|
|
|
246
253
|
**"Run codex through selftune"**
|
|
247
|
-
> Use `selftune wrap-codex -- <codex args>` instead of `codex exec <args>` directly.
|
|
254
|
+
> Use `selftune ingest wrap-codex -- <codex args>` instead of `codex exec <args>` directly.
|
|
248
255
|
|
|
249
256
|
**"Batch ingest vs real-time"**
|
|
250
|
-
> Use `selftune ingest
|
|
251
|
-
> Use `selftune wrap-codex` for ongoing sessions. Both produce the same log format.
|
|
257
|
+
> Use `selftune ingest codex` or `selftune ingest opencode` for historical sessions.
|
|
258
|
+
> Use `selftune ingest wrap-codex` for ongoing sessions. Both produce the same log format.
|
|
252
259
|
|
|
253
260
|
**"How do I know it worked?"**
|
|
254
261
|
> Run `selftune doctor` after ingestion. Check that log files exist and are parseable.
|
|
255
|
-
> Run `selftune
|
|
262
|
+
> Run `selftune eval generate --list-skills` to see if the ingested sessions appear.
|
|
@@ -4,9 +4,9 @@ Bootstrap selftune for first-time use or after changing environments.
|
|
|
4
4
|
|
|
5
5
|
## When to Use
|
|
6
6
|
|
|
7
|
-
-
|
|
8
|
-
-
|
|
9
|
-
-
|
|
7
|
+
- The user asks to set up selftune, configure selftune, or initialize selftune
|
|
8
|
+
- The agent detects `~/.selftune/config.json` does not exist
|
|
9
|
+
- The user has switched agent platforms (Claude Code, Codex, OpenCode)
|
|
10
10
|
|
|
11
11
|
## Default Command
|
|
12
12
|
|
|
@@ -77,11 +77,20 @@ Skip to Step 8 (verify with doctor) unless the user wants to reinitialize.
|
|
|
77
77
|
selftune init
|
|
78
78
|
```
|
|
79
79
|
|
|
80
|
-
### 4.
|
|
80
|
+
### 4. Hooks (Claude Code)
|
|
81
81
|
|
|
82
|
-
|
|
83
|
-
`skill/settings_snippet.json` into
|
|
84
|
-
|
|
82
|
+
Hooks are **automatically installed** by `selftune init`. The init command
|
|
83
|
+
merges selftune hook entries from `skill/settings_snippet.json` into
|
|
84
|
+
`~/.claude/settings.json` without overwriting existing user hooks. If the
|
|
85
|
+
hooks are already present, they are skipped (no duplicates).
|
|
86
|
+
|
|
87
|
+
The init output will report what was installed, e.g.:
|
|
88
|
+
|
|
89
|
+
```text
|
|
90
|
+
[INFO] Installed 4 selftune hook(s) into ~/.claude/settings.json: UserPromptSubmit, PreToolUse, PostToolUse, Stop
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
**Hook reference** (for troubleshooting):
|
|
85
94
|
|
|
86
95
|
| Hook | Script | Purpose |
|
|
87
96
|
|------|--------|---------|
|
|
@@ -92,15 +101,12 @@ are required:
|
|
|
92
101
|
| `PostToolUse` (Read) | `hooks/skill-eval.ts` | Track skill triggers |
|
|
93
102
|
| `Stop` | `hooks/session-stop.ts` | Capture session telemetry |
|
|
94
103
|
|
|
95
|
-
Derive the hook script paths from the `cli_path` field in `~/.selftune/config.json`.
|
|
96
|
-
The hooks directory is at `dirname(cli_path)/hooks/`.
|
|
97
|
-
|
|
98
104
|
**Codex agents:**
|
|
99
|
-
- Use `wrap-codex` for real-time telemetry capture (see `Workflows/Ingest.md`)
|
|
100
|
-
- Or batch-ingest existing sessions with `selftune ingest
|
|
105
|
+
- Use `selftune ingest wrap-codex` for real-time telemetry capture (see `Workflows/Ingest.md`)
|
|
106
|
+
- Or batch-ingest existing sessions with `selftune ingest codex`
|
|
101
107
|
|
|
102
108
|
**OpenCode agents:**
|
|
103
|
-
- Use `selftune ingest
|
|
109
|
+
- Use `selftune ingest opencode` to import sessions from the SQLite database
|
|
104
110
|
- See `Workflows/Ingest.md` for details
|
|
105
111
|
|
|
106
112
|
### 5. Initialize Memory Directory
|
|
@@ -121,11 +127,9 @@ watch, and rollback workflows. The directory just needs to exist.
|
|
|
121
127
|
|
|
122
128
|
### 6. Set Up Activation Rules
|
|
123
129
|
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
cp templates/activation-rules-default.json ~/.selftune/activation-rules.json
|
|
128
|
-
```
|
|
130
|
+
`selftune init` copies the default activation rules template to
|
|
131
|
+
`~/.selftune/activation-rules.json` automatically. If the file is missing,
|
|
132
|
+
run `selftune init --force` to regenerate it.
|
|
129
133
|
|
|
130
134
|
The activation rules file configures auto-activation behavior -- which skills
|
|
131
135
|
get suggested and under what conditions. Edit `~/.selftune/activation-rules.json`
|
|
@@ -133,16 +137,17 @@ to customize thresholds and skill mappings for your project.
|
|
|
133
137
|
|
|
134
138
|
### 7. Verify Agent Availability
|
|
135
139
|
|
|
136
|
-
|
|
140
|
+
`selftune init` installs the specialized agent files to `~/.claude/agents/`
|
|
141
|
+
automatically. Verify they are present:
|
|
137
142
|
|
|
138
143
|
```bash
|
|
139
|
-
ls
|
|
144
|
+
ls ~/.claude/agents/
|
|
140
145
|
```
|
|
141
146
|
|
|
142
147
|
Expected agents: `diagnosis-analyst.md`, `pattern-analyst.md`,
|
|
143
148
|
`evolution-reviewer.md`, `integration-guide.md`. These are used by evolve
|
|
144
|
-
and doctor workflows for deeper analysis. If missing,
|
|
145
|
-
|
|
149
|
+
and doctor workflows for deeper analysis. If missing, run `selftune init --force`
|
|
150
|
+
to reinstall them.
|
|
146
151
|
|
|
147
152
|
### 8. Verify with Doctor
|
|
148
153
|
|
|
@@ -163,15 +168,24 @@ Templates for each project type are in the `templates/` directory:
|
|
|
163
168
|
- `templates/multi-skill-settings.json` — hooks for multi-skill projects with activation rules
|
|
164
169
|
- `templates/activation-rules-default.json` — default auto-activation rule configuration
|
|
165
170
|
|
|
171
|
+
## Subagent Escalation
|
|
172
|
+
|
|
173
|
+
For complex project structures (monorepos, multi-skill repos, mixed agent
|
|
174
|
+
platforms), spawn the `integration-guide` agent as a subagent for guided
|
|
175
|
+
setup. This agent handles project-type detection, per-package configuration,
|
|
176
|
+
and verification steps that go beyond what the basic init workflow covers.
|
|
177
|
+
|
|
166
178
|
## Common Patterns
|
|
167
179
|
|
|
168
|
-
**
|
|
169
|
-
>
|
|
170
|
-
> install
|
|
180
|
+
**User asks to set up or initialize selftune**
|
|
181
|
+
> Run `which selftune` to check installation. If missing, install with
|
|
182
|
+
> `npm install -g selftune`. Run `selftune init`, then verify with
|
|
183
|
+
> `selftune doctor`. Report results to the user.
|
|
171
184
|
|
|
172
|
-
**
|
|
173
|
-
> Run `selftune doctor` to check hook installation.
|
|
174
|
-
>
|
|
185
|
+
**Hooks not capturing data**
|
|
186
|
+
> Run `selftune doctor` to check hook installation. Parse the JSON output
|
|
187
|
+
> for failed hook checks. If paths are wrong, update
|
|
188
|
+
> `~/.claude/settings.json` to point to actual files.
|
|
175
189
|
|
|
176
|
-
**
|
|
177
|
-
> Run `selftune init --force` to reinitialize.
|
|
190
|
+
**Config exists but appears stale**
|
|
191
|
+
> Run `selftune init --force` to reinitialize. Verify with `selftune doctor`.
|
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
# selftune Orchestrate Workflow
|
|
2
|
+
|
|
3
|
+
Run the autonomy-first selftune loop in one command.
|
|
4
|
+
|
|
5
|
+
`selftune orchestrate` is the primary closed-loop entrypoint. It runs
|
|
6
|
+
source-truth sync, computes current skill health, selects candidates,
|
|
7
|
+
deploys validated low-risk description changes autonomously, and watches
|
|
8
|
+
recent changes with auto-rollback enabled.
|
|
9
|
+
|
|
10
|
+
## When to Use
|
|
11
|
+
|
|
12
|
+
- You want the full autonomous loop, not isolated subcommands
|
|
13
|
+
- You want to improve skills without manually chaining `sync`, `status`, `evolve`, and `watch`
|
|
14
|
+
- You want a dry-run of what selftune would change next
|
|
15
|
+
- You want a stricter review policy for a single run
|
|
16
|
+
|
|
17
|
+
## Default Command
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
selftune orchestrate
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Flags
|
|
24
|
+
|
|
25
|
+
| Flag | Description | Default |
|
|
26
|
+
|------|-------------|---------|
|
|
27
|
+
| `--dry-run` | Plan and validate without deploying changes | Off |
|
|
28
|
+
| `--review-required` | Keep validated changes in review mode instead of deploying | Off |
|
|
29
|
+
| `--skill <name>` | Limit the loop to one skill | All skills |
|
|
30
|
+
| `--max-skills <n>` | Cap how many candidates are processed in one run | `3` |
|
|
31
|
+
| `--recent-window <hours>` | Window for post-deploy watch/rollback checks | `24` |
|
|
32
|
+
| `--sync-force` | Force a full source replay before candidate selection | Off |
|
|
33
|
+
|
|
34
|
+
## Default Behavior
|
|
35
|
+
|
|
36
|
+
- Sync source-truth telemetry first
|
|
37
|
+
- Prioritize critical/warning/ungraded skills with real missed-query signal
|
|
38
|
+
- Deploy validated low-risk description changes automatically
|
|
39
|
+
- Watch recent deployments and roll back regressions automatically
|
|
40
|
+
|
|
41
|
+
Use `--review-required` only when you want a stricter policy for a specific run.
|
|
42
|
+
|
|
43
|
+
## Common Patterns
|
|
44
|
+
|
|
45
|
+
**User asks to improve skills or run the full loop**
|
|
46
|
+
> Run `selftune orchestrate`. Parse the JSON output from stdout and the
|
|
47
|
+
> phased report from stderr. Report the summary to the user.
|
|
48
|
+
|
|
49
|
+
**User wants to preview changes before deploying**
|
|
50
|
+
> Run `selftune orchestrate --dry-run`. Report the planned actions without
|
|
51
|
+
> making any changes.
|
|
52
|
+
|
|
53
|
+
**User wants to focus on a single skill**
|
|
54
|
+
> Run `selftune orchestrate --skill <name>`. This limits the loop to the
|
|
55
|
+
> specified skill only.
|
|
56
|
+
|
|
57
|
+
**User wants manual review before deployment**
|
|
58
|
+
> Run `selftune orchestrate --review-required`. Validated changes stay in
|
|
59
|
+
> review mode instead of auto-deploying.
|
|
60
|
+
|
|
61
|
+
**Agent needs fresh source data before orchestrating**
|
|
62
|
+
> Run `selftune orchestrate --sync-force`. This forces a full source replay
|
|
63
|
+
> before candidate selection.
|
|
64
|
+
|
|
65
|
+
## Output
|
|
66
|
+
|
|
67
|
+
### Human-readable report (stderr)
|
|
68
|
+
|
|
69
|
+
A phased decision report printed to stderr so you can see exactly what happened and why:
|
|
70
|
+
|
|
71
|
+
1. **Phase 1: Sync** — which sources were scanned, how many records synced, repair counts
|
|
72
|
+
2. **Phase 2: Status** — skill count, system health, breakdown by status category
|
|
73
|
+
3. **Phase 3: Skill Decisions** — each skill with its action (EVOLVE / WATCH / SKIP) and reason
|
|
74
|
+
4. **Phase 4: Evolution Results** — validation pass-rate changes (before → after), deployment status
|
|
75
|
+
5. **Phase 5: Watch** — post-deploy monitoring with alert and rollback indicators
|
|
76
|
+
6. **Summary** — evaluated/deployed/watched/skipped counts and elapsed time
|
|
77
|
+
|
|
78
|
+
A mode banner at the top shows DRY RUN, REVIEW, or AUTONOMOUS with rerun hints when applicable.
|
|
79
|
+
|
|
80
|
+
### JSON output (stdout)
|
|
81
|
+
|
|
82
|
+
Machine-readable JSON with the summary fields plus a `decisions` array containing per-skill:
|
|
83
|
+
|
|
84
|
+
- `skill`, `action`, `reason`
|
|
85
|
+
- `deployed`, `evolveReason`, `validation` (before/after pass rates, improved flag) — when evolved
|
|
86
|
+
- `alert`, `rolledBack`, `passRate`, `recommendation` — when watched
|
|
87
|
+
|
|
88
|
+
This is the recommended runtime for recurring autonomous scheduling.
|
|
89
|
+
|
|
90
|
+
## Two Execution Contexts
|
|
91
|
+
|
|
92
|
+
`selftune orchestrate` runs in two contexts with different callers:
|
|
93
|
+
|
|
94
|
+
| Context | Caller | Token cost | When |
|
|
95
|
+
|---------|--------|------------|------|
|
|
96
|
+
| **Interactive** | Agent (user says "improve my skills") | Uses agent subscription | On demand |
|
|
97
|
+
| **Automated (cron)** | OS scheduler (cron/launchd/systemd) | No agent session; LLM cost only if evolution triggers | Every 6 hours |
|
|
98
|
+
| **Automated (loop)** | `selftune orchestrate --loop` | No agent session; LLM cost only if evolution triggers | Configurable interval |
|
|
99
|
+
|
|
100
|
+
In automated mode, the OS calls the CLI binary directly. No agent session
|
|
101
|
+
is created. LLM calls only happen during the evolution step (proposing and
|
|
102
|
+
validating description changes), which uses the configured model tier.
|
|
103
|
+
The orchestrate logic itself (sync, status, candidate selection) is pure
|
|
104
|
+
data processing with zero token cost.
|
|
105
|
+
|
|
106
|
+
**Cron mode:** Install OS-level scheduling with `selftune cron setup`.
|
|
107
|
+
Runs as separate invocations on a schedule (default: every 6 hours).
|
|
108
|
+
|
|
109
|
+
**Loop mode:** Run `selftune orchestrate --loop` for a long-running process
|
|
110
|
+
that cycles continuously. Use `--loop-interval <seconds>` to set the pause
|
|
111
|
+
between cycles (default: 3600s / 1 hour, minimum: 60s). Stop with Ctrl+C
|
|
112
|
+
or SIGTERM — the current cycle finishes before exit.
|
|
113
|
+
|
|
114
|
+
### Signal-Reactive Trigger
|
|
115
|
+
|
|
116
|
+
When improvement signals are detected during a session (corrections, explicit
|
|
117
|
+
requests, manual invocations), the `session-stop` hook automatically spawns a
|
|
118
|
+
focused `selftune orchestrate --max-skills 2` run in the background. This
|
|
119
|
+
reactive path complements the scheduled cron/loop modes by responding to signals
|
|
120
|
+
immediately after the session that produced them.
|
|
121
|
+
|
|
122
|
+
Guard rails:
|
|
123
|
+
- Only spawns if unconsumed signals exist in `improvement_signals.jsonl`
|
|
124
|
+
- Respects the orchestrate lock file — skips if another run started within 30 minutes
|
|
125
|
+
- Fire-and-forget: the hook exits immediately, orchestrate runs independently
|
|
126
|
+
- Silent failure: any error is swallowed so the hook never blocks Claude
|
|
127
|
+
|
|
128
|
+
### Internal Workflow Chain (Autonomous Mode)
|
|
129
|
+
|
|
130
|
+
In autonomous mode, orchestrate calls sub-workflows in this fixed order:
|
|
131
|
+
|
|
132
|
+
1. **Sync** — refresh source-truth telemetry across all supported agents (`selftune sync`)
|
|
133
|
+
2. **Status** — compute skill health using existing grade results (reads `grading.json` outputs from previous sessions)
|
|
134
|
+
3. **Evolve** — run evolution on selected candidates (pre-flight is skipped, cheap-loop mode enabled, defaults used)
|
|
135
|
+
4. **Watch** — monitor recently evolved skills (auto-rollback enabled by default, `--recent-window` hours lookback)
|
|
136
|
+
|
|
137
|
+
All sub-workflows run with defaults and no user interaction. The safety
|
|
138
|
+
model relies on regression thresholds, automatic rollback, and SKILL.md
|
|
139
|
+
backups rather than human confirmation.
|
|
@@ -1,24 +1,28 @@
|
|
|
1
|
-
# selftune
|
|
1
|
+
# selftune Ingest (Claude) Workflow
|
|
2
|
+
|
|
3
|
+
> **Note:** This workflow documents `selftune ingest claude`. The command was
|
|
4
|
+
> renamed from `selftune replay` to `selftune ingest claude`. This file is
|
|
5
|
+
> kept as `Replay.md` for routing compatibility.
|
|
2
6
|
|
|
3
7
|
Backfill the shared JSONL logs from existing Claude Code conversation
|
|
4
8
|
transcripts. Useful for bootstrapping selftune with historical session data.
|
|
5
9
|
|
|
6
10
|
## When to Use
|
|
7
11
|
|
|
8
|
-
-
|
|
9
|
-
-
|
|
10
|
-
-
|
|
12
|
+
- The user has a new selftune installation with months of Claude Code history
|
|
13
|
+
- The user re-initialized logs and wants to recover data
|
|
14
|
+
- The agent needs to populate eval data without waiting for new sessions
|
|
11
15
|
|
|
12
16
|
## Key Difference from Hooks
|
|
13
17
|
|
|
14
|
-
Real-time hooks capture only the **last** user query per session.
|
|
18
|
+
Real-time hooks capture only the **last** user query per session. Ingest
|
|
15
19
|
extracts **all** user queries, writing one `QueryLogRecord` per message.
|
|
16
20
|
This produces much richer eval data from historical sessions.
|
|
17
21
|
|
|
18
22
|
## Default Command
|
|
19
23
|
|
|
20
24
|
```bash
|
|
21
|
-
selftune
|
|
25
|
+
selftune ingest claude
|
|
22
26
|
```
|
|
23
27
|
|
|
24
28
|
## Options
|
|
@@ -50,21 +54,38 @@ which transcripts have already been ingested. Use `--force` to re-ingest all.
|
|
|
50
54
|
|
|
51
55
|
## Steps
|
|
52
56
|
|
|
53
|
-
1.
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
+
### 1. Preview Ingestion
|
|
58
|
+
|
|
59
|
+
Run `selftune ingest claude --dry-run`. Parse the output to check how many
|
|
60
|
+
transcripts would be ingested. Report the count to the user.
|
|
61
|
+
|
|
62
|
+
### 2. Run Ingestion
|
|
63
|
+
|
|
64
|
+
Run `selftune ingest claude`. Parse the output for ingested session counts
|
|
65
|
+
and any errors.
|
|
66
|
+
|
|
67
|
+
### 3. Verify Results
|
|
68
|
+
|
|
69
|
+
Run `selftune doctor` to verify logs are healthy. Run
|
|
70
|
+
`selftune eval generate --list-skills` to confirm ingested sessions appear.
|
|
71
|
+
|
|
72
|
+
### 4. Report Results
|
|
73
|
+
|
|
74
|
+
Report the number of sessions ingested and any skills discovered to the user.
|
|
57
75
|
|
|
58
76
|
## Common Patterns
|
|
59
77
|
|
|
60
|
-
**
|
|
61
|
-
> Run `selftune
|
|
78
|
+
**User wants to backfill logs from Claude Code history**
|
|
79
|
+
> Run `selftune ingest claude`. No options needed for a full backfill.
|
|
80
|
+
> Parse the output and report ingested session counts.
|
|
62
81
|
|
|
63
|
-
**
|
|
64
|
-
> Run `selftune
|
|
82
|
+
**User wants to ingest only recent sessions**
|
|
83
|
+
> Run `selftune ingest claude --since <date>` with the user's specified date.
|
|
65
84
|
|
|
66
|
-
**
|
|
67
|
-
> Run `selftune
|
|
85
|
+
**User wants to re-ingest everything from scratch**
|
|
86
|
+
> Run `selftune ingest claude --force`. This ignores the marker file and
|
|
87
|
+
> rescans all transcripts.
|
|
68
88
|
|
|
69
|
-
**
|
|
70
|
-
> Run `selftune doctor` after
|
|
89
|
+
**Agent needs to verify ingestion succeeded**
|
|
90
|
+
> Run `selftune doctor` after ingestion. Parse the JSON output to check
|
|
91
|
+
> that log file entry counts increased.
|
|
@@ -6,7 +6,7 @@ Records the rollback in the evolution audit log for traceability.
|
|
|
6
6
|
## Default Command
|
|
7
7
|
|
|
8
8
|
```bash
|
|
9
|
-
selftune rollback --skill <name> --skill-path <path> [options]
|
|
9
|
+
selftune evolve rollback --skill <name> --skill-path <path> [options]
|
|
10
10
|
```
|
|
11
11
|
|
|
12
12
|
## Options
|
|
@@ -95,13 +95,13 @@ If `--proposal-id` is specified, use that instead.
|
|
|
95
95
|
### 2. Run Rollback
|
|
96
96
|
|
|
97
97
|
```bash
|
|
98
|
-
selftune rollback --skill pptx --skill-path /path/to/SKILL.md
|
|
98
|
+
selftune evolve rollback --skill pptx --skill-path /path/to/SKILL.md
|
|
99
99
|
```
|
|
100
100
|
|
|
101
101
|
Or to rollback a specific proposal:
|
|
102
102
|
|
|
103
103
|
```bash
|
|
104
|
-
selftune rollback --skill pptx --skill-path /path/to/SKILL.md --proposal-id evolve-pptx-1709125200000
|
|
104
|
+
selftune evolve rollback --skill pptx --skill-path /path/to/SKILL.md --proposal-id evolve-pptx-1709125200000
|
|
105
105
|
```
|
|
106
106
|
|
|
107
107
|
### 3. Verify Restoration
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# selftune Schedule Workflow
|
|
2
|
+
|
|
3
|
+
Generate ready-to-use scheduling examples for automating selftune with
|
|
4
|
+
standard system tools. This is the **primary automation path** — it works
|
|
5
|
+
on any machine without requiring a specific agent runtime.
|
|
6
|
+
|
|
7
|
+
For OpenClaw-specific scheduling, see `Workflows/Cron.md`.
|
|
8
|
+
|
|
9
|
+
## When to Use
|
|
10
|
+
|
|
11
|
+
- Setting up selftune automation for the first time
|
|
12
|
+
- Generating crontab entries for a Linux/macOS server
|
|
13
|
+
- Creating a launchd plist for a macOS machine
|
|
14
|
+
- Creating a systemd timer for a Linux server
|
|
15
|
+
- Understanding the selftune automation loop
|
|
16
|
+
|
|
17
|
+
## The Automation Loop
|
|
18
|
+
|
|
19
|
+
The core selftune automation loop is one command:
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
selftune orchestrate
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
`selftune orchestrate` runs source-truth sync first, selects candidate skills,
|
|
26
|
+
deploys validated low-risk description changes autonomously, and watches recent
|
|
27
|
+
deployments with auto-rollback enabled.
|
|
28
|
+
|
|
29
|
+
## Default Command
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
selftune schedule
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Outputs examples for all three scheduling systems (cron, launchd, systemd).
|
|
36
|
+
|
|
37
|
+
## Flags
|
|
38
|
+
|
|
39
|
+
| Flag | Description | Default |
|
|
40
|
+
|------|-------------|---------|
|
|
41
|
+
| `--format <type>` | Output only one format: `cron`, `launchd`, or `systemd` | All formats |
|
|
42
|
+
| `--install` | Write and activate scheduler artifacts for the selected/default platform | Off |
|
|
43
|
+
| `--dry-run` | Preview installed files and activation commands without writing | Off |
|
|
44
|
+
| `--help` | Show help message | — |
|
|
45
|
+
|
|
46
|
+
## Steps
|
|
47
|
+
|
|
48
|
+
1. Run `selftune schedule` to see all examples
|
|
49
|
+
2. Pick the scheduling system for your platform
|
|
50
|
+
3. Install them directly with `--install`, or inspect/customize the raw snippets first
|
|
51
|
+
|
|
52
|
+
## Alias
|
|
53
|
+
|
|
54
|
+
`selftune schedule` is now an alias for `selftune cron`. Both commands are interchangeable. See `Workflows/Cron.md` for the full cron workflow reference.
|
|
55
|
+
|
|
56
|
+
## Common Patterns
|
|
57
|
+
|
|
58
|
+
- **User wants quick setup on a Linux server** -- Run `selftune schedule --install --format cron`.
|
|
59
|
+
- **User wants setup on macOS** -- Run `selftune schedule --install --format launchd`.
|
|
60
|
+
- **User wants setup on a systemd-based server** -- Run `selftune schedule --install --format systemd`.
|
|
61
|
+
- **User mentions OpenClaw** -- Use `selftune cron setup --platform openclaw` for the OpenClaw scheduler adapter. The default product path is still `selftune schedule --install`. See `Workflows/Cron.md`.
|