selftune 0.2.9 → 0.2.10

package/skill/Workflows/Sync.md

@@ -19,21 +19,22 @@ selftune sync
 
 ## Options
 
-| Flag | Description |
-|------|-------------|
-| `--since <date>` | Only sync sessions modified on/after this date |
-| `--dry-run` | Show summary without writing files |
-| `--force` | Ignore per-source markers and rescan everything |
-| `--no-claude` | Skip Claude transcript replay |
-| `--no-codex` | Skip Codex rollout ingest |
-| `--no-opencode` | Skip OpenCode ingest |
-| `--no-openclaw` | Skip OpenClaw ingest |
-| `--no-repair` | Skip rebuilding `skill_usage_repaired.jsonl` |
-| `--json` | Output results as JSON |
+| Flag             | Description                                     |
+| ---------------- | ----------------------------------------------- |
+| `--since <date>` | Only sync sessions modified on/after this date  |
+| `--dry-run`      | Show summary without writing files              |
+| `--force`        | Ignore per-source markers and rescan everything |
+| `--no-claude`    | Skip Claude transcript replay                   |
+| `--no-codex`     | Skip Codex rollout ingest                       |
+| `--no-opencode`  | Skip OpenCode ingest                            |
+| `--no-openclaw`  | Skip OpenClaw ingest                            |
+| `--no-repair`    | Skip rebuilding `skill_usage_repaired.jsonl`    |
+| `--json`         | Output results as JSON                          |
 
 ## Output
 
 Writes/refreshed data:
+
 - `~/.claude/session_telemetry_log.jsonl`
 - `~/.claude/all_queries_log.jsonl`
 - `~/.claude/skill_usage_log.jsonl`
@@ -50,6 +51,7 @@ counts. Report the preview summary to the user.
 ### 2. Run Sync
 
 Run `selftune sync`. The output includes:
+
 - Per-source `scanned`, `synced`, and `skipped` counts
 - Repaired overlay totals
 - Any errors or warnings
@@ -92,20 +94,25 @@ Use `--json` when the agent needs to parse sync results programmatically
 ## Common Patterns
 
 **User wants to refresh telemetry data**
+
 > Run `selftune sync`. Report per-source `scanned`, `synced`, and `skipped` counts.
 
 **User wants to sync only recent sessions**
+
 > Run `selftune sync --since <date>` with the user's specified date.
 
 **User wants a full rescan from scratch**
+
 > Run `selftune sync --force`. This ignores per-source markers and rescans
 > all sessions.
 
 **Agent needs to verify sync worked**
+
 > Check per-source `scanned`, `synced`, and `skipped` counts. `synced=0`
 > is normal when data is already up-to-date. Verify `scanned > 0` for
 > expected sources to confirm sync ran successfully.
 
 **Agent is chaining into monitoring or evolution**
+
 > Use `selftune watch --sync-first` or `selftune evolve --sync-first` to
 > refresh source truth automatically before making decisions.

package/skill/Workflows/UnitTest.md

@@ -11,15 +11,15 @@ selftune eval unit-test --skill <name> --tests <path> [options]
 
 ## Options
 
-| Flag | Description | Default |
-|------|-------------|---------|
-| `--skill <name>` | Skill name | Required |
-| `--tests <path>` | Path to unit test JSON file | `~/.selftune/unit-tests/<skill>.json` |
-| `--run-agent` | Run agent-based assertions (not just trigger checks) | Off |
-| `--generate` | Generate tests from skill content instead of running | Off |
-| `--skill-path <path>` | Path to SKILL.md (required for `--generate`) | None |
-| `--eval-set <path>` | Eval set for failure context (used with `--generate`) | None |
-| `--model <flag>` | Model flag for LLM calls | Agent default |
+| Flag                  | Description                                           | Default                               |
+| --------------------- | ----------------------------------------------------- | ------------------------------------- |
+| `--skill <name>`      | Skill name                                            | Required                              |
+| `--tests <path>`      | Path to unit test JSON file                           | `~/.selftune/unit-tests/<skill>.json` |
+| `--run-agent`         | Run agent-based assertions (not just trigger checks)  | Off                                   |
+| `--generate`          | Generate tests from skill content instead of running  | Off                                   |
+| `--skill-path <path>` | Path to SKILL.md (required for `--generate`)          | None                                  |
+| `--eval-set <path>`   | Eval set for failure context (used with `--generate`) | None                                  |
+| `--model <flag>`      | Model flag for LLM calls                              | Agent default                         |
 
 ## Test Format
 
@@ -48,12 +48,12 @@ Tests are stored as JSON arrays in `~/.selftune/unit-tests/<skill>.json`:
 
 ## Assertion Types
 
-| Type | What it checks | Requires agent? |
-|------|---------------|-----------------|
-| `trigger_check` | Query triggers the skill description | No (LLM only) |
-| `output_contains` | Agent output contains expected text | Yes |
-| `output_matches_regex` | Agent output matches regex pattern | Yes |
-| `tool_called` | Agent used a specific tool | Yes |
+| Type                   | What it checks                       | Requires agent? |
+| ---------------------- | ------------------------------------ | --------------- |
+| `trigger_check`        | Query triggers the skill description | No (LLM only)   |
+| `output_contains`      | Agent output contains expected text  | Yes             |
+| `output_matches_regex` | Agent output matches regex pattern   | Yes             |
+| `tool_called`          | Agent used a specific tool           | Yes             |
 
 Trigger check assertions are cheap (single LLM call). Agent-based assertions
 require `--run-agent` and run the query through the full agent.
@@ -66,14 +66,19 @@ require `--run-agent` and run the query through the full agent.
   "total": 10,
   "passed": 8,
   "failed": 2,
-  "pass_rate": 0.80,
+  "pass_rate": 0.8,
   "results": [
     {
       "test_id": "research-trigger-1",
       "overall_passed": true,
       "trigger_passed": true,
       "assertion_results": [
-        { "type": "trigger_check", "value": "true", "passed": true, "evidence": "LLM responded YES" }
+        {
+          "type": "trigger_check",
+          "value": "true",
+          "passed": true,
+          "evidence": "LLM responded YES"
+        }
       ],
       "duration_ms": 450
     }
@@ -93,6 +98,7 @@ selftune eval unit-test --skill Research --generate --skill-path ~/.claude/skill
 ```
 
 Parse the output. The LLM creates test cases covering:
+
 - Explicit trigger queries
 - Implicit trigger queries
 - Contextual trigger queries
@@ -114,6 +120,7 @@ Add `--run-agent` for full agent-based assertions.
 ### 3. Parse Results
 
 Parse the JSON output. Check `pass_rate` and investigate failures:
+
 - Failed trigger checks -- description needs improvement (route to Evolve)
 - Failed output assertions -- skill workflow needs fixes
 - Failed tool assertions -- skill routing is broken
@@ -134,17 +141,21 @@ the evolution improved trigger accuracy.
 ## Common Patterns
 
 **User asks to generate tests for a skill**
+
 > Run `selftune eval unit-test --skill <name> --generate --skill-path <path>`.
 > Parse the output and report how many tests were generated.
 
 **User asks to run existing tests**
+
 > Run `selftune eval unit-test --skill <name>`. Parse the JSON output and
 > report pass rate and any failures.
 
 **User asks for full agent-based testing**
+
 > Run `selftune eval unit-test --skill <name> --run-agent`. This runs queries
 > through the full agent, so inform the user it will take longer.
 
 **After an evolution completes**
+
 > Run unit tests to verify the evolution improved trigger accuracy. Compare
 > the new pass rate against the pre-evolution baseline.

package/skill/Workflows/Watch.md

@@ -11,15 +11,15 @@ selftune watch --skill <name> --skill-path <path> [options]
 
 ## Options
 
-| Flag | Description | Default |
-|------|-------------|---------|
-| `--skill <name>` | Skill name | Required |
-| `--skill-path <path>` | Path to the skill's SKILL.md | Required |
-| `--window <n>` | Sliding window size (number of sessions) | 20 |
-| `--threshold <n>` | Regression threshold (drop from baseline) | 0.1 |
-| `--auto-rollback` | Automatically rollback on detected regression | Off |
-| `--sync-first` | Refresh source-truth telemetry before evaluating | Off |
-| `--sync-force` | Force a full source rescan during `--sync-first` | Off |
+| Flag                  | Description                                      | Default  |
+| --------------------- | ------------------------------------------------ | -------- |
+| `--skill <name>`      | Skill name                                       | Required |
+| `--skill-path <path>` | Path to the skill's SKILL.md                     | Required |
+| `--window <n>`        | Sliding window size (number of sessions)         | 20       |
+| `--threshold <n>`     | Regression threshold (drop from baseline)        | 0.1      |
+| `--auto-rollback`     | Automatically rollback on detected regression    | Off      |
+| `--sync-first`        | Refresh source-truth telemetry before evaluating | Off      |
+| `--sync-force`        | Force a full source rescan during `--sync-first` | Off      |
 
 ## Output Format
 
@@ -40,12 +40,12 @@ selftune watch --skill <name> --skill-path <path> [options]
 
 ### Status Values
 
-| Status | Meaning |
-|--------|---------|
-| `healthy` | Current pass rate is within threshold of baseline |
-| `warning` | Pass rate dropped but within threshold |
-| `regression` | Pass rate dropped below baseline minus threshold |
-| `insufficient_data` | Not enough sessions in the window to evaluate |
+| Status              | Meaning                                           |
+| ------------------- | ------------------------------------------------- |
+| `healthy`           | Current pass rate is within threshold of baseline |
+| `warning`           | Pass rate dropped but within threshold            |
+| `regression`        | Pass rate dropped below baseline minus threshold  |
+| `insufficient_data` | Not enough sessions in the window to evaluate     |
 
 ## Parsing Instructions
 
@@ -69,6 +69,7 @@ selftune watch --skill <name> --skill-path <path> [options]
 ### 0. Read Evolution Context
 
 Read `~/.selftune/memory/context.md` for session context:
+
 - Active evolutions and their current status
 - Known issues and regression history
 - Last update timestamp
@@ -91,16 +92,17 @@ selftune watch --skill pptx --skill-path /path/to/SKILL.md
 
 Parse the JSON output. Key decision points:
 
-| Status | Action |
-|--------|--------|
-| `healthy` | No action needed. Skill is performing well. |
-| `warning` | Monitor closely. Consider re-running after more sessions. |
-| `regression` | Investigate. Consider rollback. |
-| `insufficient_data` | Wait for more sessions before evaluating. |
+| Status              | Action                                                    |
+| ------------------- | --------------------------------------------------------- |
+| `healthy`           | No action needed. Skill is performing well.               |
+| `warning`           | Monitor closely. Consider re-running after more sessions. |
+| `regression`        | Investigate. Consider rollback.                           |
+| `insufficient_data` | Wait for more sessions before evaluating.                 |
 
 ### 3. Decide Action
 
 If regression is detected:
+
 - Review recent session transcripts to understand what changed
 - Check if the eval set is still representative
 - Run `evolve rollback` if the regression is confirmed (see `Workflows/Rollback.md`)
@@ -111,6 +113,7 @@ previous description and logs a `rolled_back` entry.
 ### 4. Report
 
 Summarize the snapshot for the user:
+
 - Current pass rate vs baseline
 - Number of sessions evaluated
 - Whether regression was detected
@@ -126,16 +129,20 @@ context window resets before the user acts on the results.
 ## Common Patterns
 
 **"Is the skill performing well after the change?"**
+
 > Run watch with the skill name and path. Report the snapshot.
 
 **"Check for regressions"**
+
 > Same as above. Focus on the `regression_detected` and `delta` fields.
 
 **"How is the skill doing?"**
+
 > Run watch. If `insufficient_data`, tell the user to wait for more
 > sessions before drawing conclusions.
 
 **"Auto-rollback if it regresses"**
+
 > Use `--auto-rollback`. The command will restore the previous description
 > automatically if pass rate drops below baseline minus threshold.
 

package/skill/agents/diagnosis-analyst.md

@@ -88,6 +88,7 @@ selftune eval generate --skill <name> --max 50
 
 Treat these outputs as exploratory summaries. Verify important claims against
 the underlying logs:
+
 - `~/.claude/skill_usage_log.jsonl`
 - `~/.claude/all_queries_log.jsonl`
 - `~/.claude/session_telemetry_log.jsonl`
@@ -96,6 +97,7 @@ the underlying logs:
 
 Read `~/.claude/evolution_audit_log.jsonl` for entries affecting the target
 skill. Look for:
+
 - recent deploys followed by regressions
 - repeated dry-runs or validated proposals with no deploy
 - rollbacks
@@ -107,6 +109,7 @@ Prefer the specific sessions passed by the parent. Otherwise, select recent
 sessions that show errors, unmatched queries, or clear misses.
 
 Look for:
+
 - the skill never being read or invoked
 - the wrong workflow being chosen
 - steps performed out of order
@@ -121,6 +124,7 @@ smallest credible next action.
 ## Stop Conditions
 
 Stop and return to the parent if:
+
 - the target skill is ambiguous
 - the required logs or transcripts are unavailable
 - the evidence is limited to one isolated session
@@ -134,30 +138,37 @@ Return a compact report with these sections:
 ## Diagnosis Report: <skill-name>
 
 ### Summary
+
 [2-4 sentence explanation of what is going wrong]
 
 ### Root Cause
+
 [TRIGGER / PROCESS / QUALITY / INFRASTRUCTURE]
 
 ### Findings
+
 - [Finding 1]
 - [Finding 2]
 - [Finding 3]
 
 ### Evidence
+
 - [path or command result]
 - [session ID / query / timestamp]
 - [audit or transcript evidence]
 
 ### Recommended Next Actions
+
 1. [Highest-leverage next step]
 2. [Second step]
 3. [Optional follow-up]
 
 ### Suggested Commands
+
 - `...`
 - `...`
 
 ### Confidence
+
 [high / medium / low]
 ```

package/skill/agents/evolution-reviewer.md

@@ -3,7 +3,7 @@ name: evolution-reviewer
 description: Use when reviewing a dry-run or pending evolution proposal before deployment, especially for high-stakes skills, marginal improvements, or recent regressions. Compares old vs new content, checks evidence quality, and returns an approve or reject verdict with conditions.
 tools: Read, Grep, Glob, Bash
 disallowedTools: Write, Edit
-model: sonnet
+model: opus
 maxTurns: 8
 ---
 
@@ -69,12 +69,14 @@ selftune evolve --skill <name> --skill-path <path> --dry-run
 ### 2. Compare original vs proposed content
 
 For description proposals, compare:
+
 - preserved working anchors
 - added language for missed queries
 - scope creep or vague broadening
 - tone and style continuity
 
 For routing/body proposals, compare:
+
 - workflow routing ownership changes
 - added or removed operational steps
 - whether the body still matches current CLI behavior
@@ -83,6 +85,7 @@ For routing/body proposals, compare:
 ### 3. Assess eval and evidence quality
 
 Check:
+
 - eval size is meaningful for the change being proposed
 - negatives exist for overtriggering protection
 - explicit queries are protected
@@ -91,6 +94,7 @@ Check:
 ### 4. Check metrics and history
 
 Review proposal metrics and recent history:
+
 - pass-rate delta
 - regression count or obvious explicit regressions
 - confidence
@@ -99,6 +103,7 @@ Review proposal metrics and recent history:
 ### 5. Render a safety verdict
 
 Issue one of:
+
 - `APPROVE`
 - `APPROVE WITH CONDITIONS`
 - `REJECT`
@@ -106,6 +111,7 @@ Issue one of:
 ## Stop Conditions
 
 Stop and return to the parent if:
+
 - there is no concrete proposal or diff to review
 - the target skill or proposal is ambiguous
 - the eval source is missing
@@ -119,31 +125,39 @@ Return a compact verdict with these sections:
 ## Evolution Review: <skill-name>
 
 ### Proposal ID
+
 [proposal ID or "not provided"]
 
 ### Verdict
+
 [APPROVE / APPROVE WITH CONDITIONS / REJECT]
 
 ### Summary
+
 [2-4 sentence explanation]
 
 ### Findings
+
 - [Finding 1]
 - [Finding 2]
 - [Finding 3]
 
 ### Evidence
+
 - [audit entry / eval fact / diff observation]
 - [audit entry / eval fact / diff observation]
 
 ### Required Changes
+
 1. [Only if not approved]
 2. [Only if not approved]
 
 ### Post-Deploy Conditions
+
 - [watch requirement or monitoring threshold]
 - [follow-up check]
 
 ### Confidence
+
 [high / medium / low]
 ```

package/skill/agents/integration-guide.md

@@ -46,6 +46,7 @@ parent. Do not ask the user directly unless the parent explicitly told you to.
 ### 1. Detect project structure
 
 Inspect the workspace and classify it as one of:
+
 - single-skill project
 - multi-skill repo
 - monorepo with shared tooling
@@ -64,6 +65,7 @@ selftune doctor
 ```
 
 Check:
+
 - whether the CLI exists
 - whether `config.json` exists and looks current (resolve via `SELFTUNE_CONFIG_DIR` or `SELFTUNE_HOME` env vars first, falling back to `~/.selftune/`; run `selftune doctor` to confirm the resolved path)
 - whether hooks or ingest paths are healthy
@@ -80,6 +82,7 @@ selftune init [--agent claude_code] [--cli-path <path>] [--force]
 For other platforms, route to the appropriate ingest workflow after init.
 
 If the repo layout is complex, decide whether the user needs:
+
 - one shared setup at the repo root
 - per-package setup guidance
 - absolute paths to avoid cwd-dependent failures
@@ -89,6 +92,7 @@ If the repo layout is complex, decide whether the user needs:
 If `requestedMode` is `plan-only`, stop at a verified setup plan.
 
 If `requestedMode` is `hands-on`, you may:
+
 - run `selftune init`
 - create or refresh local activation-rules files
 - repair obvious path or config issues
@@ -116,6 +120,7 @@ run evals, improve a skill, or set up autonomous orchestration.
 ## Stop Conditions
 
 Stop and return to the parent if:
+
 - the project root is ambiguous
 - the CLI is missing and installation is not allowed
 - the repo has no skills and the task is really skill creation, not setup
@@ -130,25 +135,30 @@ Return a setup report with these sections:
 ## selftune Setup Complete
 
 ### Environment
+
 - Agent platform: <claude_code / codex / opencode / openclaw / unknown>
 - Project type: <single-skill / multi-skill / monorepo / no-skills>
 - Skills detected: <list>
 
 ### Configuration
+
 - Config: [created / verified / missing]
 - Init path: [command used or recommended]
 - Hooks or ingest: [healthy / needs work / not applicable]
 - Doctor: [healthy / unhealthy with blockers]
 
 ### Verification
+
 - Telemetry capture: [working / not verified]
 - Skill tracking: [working / not verified]
 
 ### Next Steps
+
 1. [Primary recommended action]
 2. [Secondary action]
 3. [Optional action]
 
 ### Confidence
+
 [high / medium / low]
 ```

package/skill/agents/pattern-analyst.md

@@ -66,6 +66,7 @@ Then read the actual `SKILL.md` files for the skills in scope.
 ### 2. Extract each skill's ownership contract
 
 For each skill, capture:
+
 - frontmatter description
 - workflow-routing triggers
 - explicit exclusions or negative examples
@@ -74,6 +75,7 @@ For each skill, capture:
 ### 3. Detect conflicts and gaps
 
 Compare trigger keywords and description phrases across all skills. Flag:
+
 - direct conflicts
 - semantic overlaps
 - negative-example gaps
@@ -83,6 +85,7 @@ Compare trigger keywords and description phrases across all skills. Flag:
 ### 4. Analyze real query behavior
 
 Read the logs and look for:
+
 - queries that triggered multiple skills
 - queries that triggered no skills despite matching one or more descriptions
 - queries that appear to have been routed to the wrong skill
@@ -103,6 +106,7 @@ shifted ownership or introduced churn.
 ### 6. Recommend ownership changes
 
 For each important conflict, state:
+
 - which skill should own the query family
 - which skill should back off
 - whether the fix is a description change, routing-table change, negative
@@ -111,6 +115,7 @@ For each important conflict, state:
 ## Stop Conditions
 
 Stop and return to the parent if:
+
 - the skills in scope are not identifiable
 - there is not enough log data to say anything useful
 - the question is really about one underperforming skill rather than
@@ -124,26 +129,32 @@ Return a compact report with these sections:
 ## Cross-Skill Pattern Analysis
 
 ### Summary
+
 [2-4 sentence overview]
 
 ### Findings
+
 - [Finding 1]
 - [Finding 2]
 - [Finding 3]
 
 ### Conflict Matrix
+
 | Skill A | Skill B | Problem | Evidence | Recommended Owner |
-|---------|---------|---------|----------|-------------------|
+| ------- | ------- | ------- | -------- | ----------------- |
 | ...     | ...     | ...     | ...      | ...               |
 
 ### Coverage Gaps
+
 - [query family or sample]
 
 ### Recommended Changes
+
 1. [Highest-priority change]
 2. [Second change]
 3. [Optional follow-up]
 
 ### Confidence
+
 [high / medium / low]
 ```

package/skill/references/grading-methodology.md

@@ -9,11 +9,11 @@ referenced by evolution workflows to understand quality signals.
 
 Every session is graded across three tiers, each answering a different question:
 
-| Tier | Question | Example expectation |
-|------|----------|---------------------|
-| **Trigger** | Did the skill fire at all? | `skills_triggered` contains the skill name |
-| **Process** | Did the agent follow the right steps? | SKILL.md was read before main work started |
-| **Quality** | Was the output actually good? | Output file has correct content and structure |
+| Tier        | Question                              | Example expectation                           |
+| ----------- | ------------------------------------- | --------------------------------------------- |
+| **Trigger** | Did the skill fire at all?            | `skills_triggered` contains the skill name    |
+| **Process** | Did the agent follow the right steps? | SKILL.md was read before main work started    |
+| **Quality** | Was the output actually good?         | Output file has correct content and structure |
 
 A session can pass Trigger but fail Process (skill fired, but steps were wrong),
 or pass Process but fail Quality (steps were right, but output was bad).
@@ -71,13 +71,14 @@ Always include at least one Process and one Quality expectation.
 After grading explicit expectations, extract 2-4 implicit claims from the transcript.
 Each claim falls into one of three types:
 
-| Type | What it captures | Example |
-|------|------------------|---------|
-| **Factual** | A verifiable statement the agent made | "The agent said 12 slides were created" |
-| **Process** | An observed behavior pattern | "The agent read SKILL.md before making any file changes" |
-| **Quality** | An output characteristic | "The output file was named correctly" |
+| Type        | What it captures                      | Example                                                  |
+| ----------- | ------------------------------------- | -------------------------------------------------------- |
+| **Factual** | A verifiable statement the agent made | "The agent said 12 slides were created"                  |
+| **Process** | An observed behavior pattern          | "The agent read SKILL.md before making any file changes" |
+| **Quality** | An output characteristic              | "The output file was named correctly"                    |
 
 For each claim:
+
 1. State the claim clearly
 2. Classify its type
 3. Mark `verified: true` or `verified: false`
@@ -153,9 +154,7 @@ Only raise things worth improving. The goal is actionable feedback, not exhausti
     }
   ],
   "eval_feedback": {
-    "suggestions": [
-      { "reason": "No expectation checks slide content" }
-    ],
+    "suggestions": [{ "reason": "No expectation checks slide content" }],
     "overall": "Process coverage good; add output quality assertions."
   }
 }
@@ -163,14 +162,14 @@ Only raise things worth improving. The goal is actionable feedback, not exhausti
 
 ### Field descriptions
 
-| Field | Type | Description |
-|-------|------|-------------|
-| `session_id` | string | From session telemetry |
-| `skill_name` | string | The skill being graded |
-| `transcript_path` | string | Path to the session transcript JSONL |
-| `graded_at` | string | ISO 8601 timestamp of grading |
-| `expectations[]` | array | Each expectation with verdict and evidence |
-| `summary` | object | Aggregate pass/fail counts and rate |
-| `execution_metrics` | object | Raw metrics from session telemetry |
-| `claims[]` | array | Implicit claims extracted from transcript |
-| `eval_feedback` | object | Suggestions for improving the eval set |
+| Field               | Type   | Description                                |
+| ------------------- | ------ | ------------------------------------------ |
+| `session_id`        | string | From session telemetry                     |
+| `skill_name`        | string | The skill being graded                     |
+| `transcript_path`   | string | Path to the session transcript JSONL       |
+| `graded_at`         | string | ISO 8601 timestamp of grading              |
+| `expectations[]`    | array  | Each expectation with verdict and evidence |
+| `summary`           | object | Aggregate pass/fail counts and rate        |
+| `execution_metrics` | object | Raw metrics from session telemetry         |
+| `claims[]`          | array  | Implicit claims extracted from transcript  |
+| `eval_feedback`     | object | Suggestions for improving the eval set     |