selftune 0.1.4 → 0.2.1

package/skill/Workflows/Baseline.md

@@ -0,0 +1,144 @@
+# selftune Baseline Workflow
+
+Measure whether a skill adds value over a no-skill baseline. Runs trigger
+checks with and without the skill description to compute lift — the
+improvement in pass rate that the skill provides.
+
+## When to Invoke
+
+Invoke this workflow when the user requests any of the following:
+- Measuring whether a skill adds value or is worth keeping
+- Comparing skill performance against a no-skill baseline
+- Deciding whether to evolve or rework a skill
+- Any request containing "baseline", "does this skill help", or "skill value"
+
+## Default Command
+
+```bash
+selftune grade baseline --skill <name> --skill-path <path> [options]
+```
+
+## Options
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--skill <name>` | Skill name | Required |
+| `--skill-path <path>` | Path to the skill's SKILL.md | Required |
+| `--eval-set <path>` | Pre-built eval set JSON | Auto-generated from logs |
+| `--agent <name>` | Agent CLI to use | Auto-detected |
+
+## Output Format
+
+```json
+{
+  "skill_name": "Research",
+  "eval_set_size": 25,
+  "baseline_pass_rate": 0.32,
+  "with_skill_pass_rate": 0.88,
+  "lift": 0.56,
+  "adds_value": true,
+  "measured_at": "2026-03-04T12:00:00.000Z"
+}
+```
+
+## How It Works
+
+1. Loads the eval set (from `--eval-set` or auto-generated from logs)
+2. Reads the skill's current description from SKILL.md
+3. Runs trigger checks against an **empty description** (no-skill baseline)
+4. Runs trigger checks against the **actual description** (with-skill)
+5. Computes pass rates for both conditions
+6. Calculates `lift = with_skill_pass_rate - baseline_pass_rate`
+7. Sets `adds_value = lift >= 0.05`
+
+## Integration with Evolve
+
+The `selftune evolve` command supports a `--with-baseline` flag:
+
+```bash
+selftune evolve --skill Research --skill-path /path/SKILL.md --with-baseline
+```
+
+When enabled, the evolve command measures baseline lift before deploying.
+If the skill doesn't add at least 5% lift over no-skill, the evolution is
+skipped — the skill needs fundamental rework, not description tweaks.
+
+## Steps
+
+### 0. Pre-Flight Configuration
+
+Before running baseline measurement, present numbered configuration options to the user inline in your response, then wait for the user's answer before proceeding.
+
+If the user responds with "use defaults", "just do it", or similar shorthand, skip to step 1 using the recommended defaults.
+
+Present the following options inline in your response:
+
+1. **Eval Set Source**
+   - a) Auto-generate from logs (recommended if logs exist)
+   - b) Use existing eval set file — provide path
+   - c) Generate synthetic evals first (for new skills with no data)
+
+2. **Agent CLI**
+   - a) Auto-detect (recommended)
+   - b) Specify: claude / codex / opencode
+
+Ask: "Reply with your choices or 'use defaults' for recommended settings."
+
+After the user responds, parse their selections and map each choice to the corresponding CLI flags:
+
+| Selection | CLI Flag |
+|-----------|----------|
+| 1a (auto-generate) | _(no flag, default)_ |
+| 1b (existing eval set) | `--eval-set <path>` |
+| 1c (synthetic first) | Run Evals workflow with `--synthetic` first, then use output |
+| 2a (auto-detect) | _(no flag, default)_ |
+| 2b (specify agent) | `--agent <name>` |
+
+Show a confirmation summary to the user:
+
+```
+Configuration Summary:
+  Eval source:   auto-generate from logs
+  Agent:         auto-detect
+
+Proceeding...
+```
+
+Build the CLI command string with all selected flags and continue to step 1.
+
+### 1. Run Baseline Measurement
+
+```bash
+selftune grade baseline --skill Research --skill-path ~/.claude/skills/Research/SKILL.md
+```
+
+Parse the JSON output and extract `lift` and `adds_value` fields.
+
+### 2. Interpret Results
+
+| Lift | Interpretation | Action |
+|------|---------------|--------|
+| >= 0.20 | Strong value | Skill is working well |
+| 0.05–0.20 | Moderate value | Consider evolving to improve |
+| < 0.05 | Minimal value | Skill may need rework, not just evolution |
+| < 0 | Negative value | Skill is hurting — investigate or disable |
+
+Report the interpretation to the user based on the lift value.
+
+### 3. Use as Evolution Gate
+
+Add `--with-baseline` to evolve commands to prevent wasting evolution
+cycles on skills that don't add value.
+
+## Common Patterns
+
+**User asks whether a skill adds value (e.g., "does the Research skill help?"):**
+Run `selftune grade baseline --skill Research --skill-path ~/.claude/skills/Research/SKILL.md`.
+Parse the JSON output and report the lift value with interpretation.
+
+**User wants to gate evolution on baseline value:**
+Run `selftune evolve --skill Research --skill-path /path/SKILL.md --with-baseline`.
+This measures baseline lift before deploying and skips evolution if lift is below 5%.
+
+**User wants to test with a custom eval set:**
+Run `selftune grade baseline --skill pptx --skill-path /path/SKILL.md --eval-set evals-pptx.json`.

package/skill/Workflows/Composability.md

@@ -0,0 +1,107 @@
+# selftune Composability Workflow
+
+Analyze how skills interact when triggered together in the same session.
+Detects conflict candidates — skill pairs that produce more errors when
+co-occurring than when used alone.
+
+## Default Command
+
+```bash
+selftune eval composability --skill <name> [options]
+```
+
+## Options
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--skill <name>` | Skill to analyze | Required |
+| `--window <n>` | Only analyze sessions from last N days | All sessions |
+| `--telemetry-log <path>` | Path to telemetry log | `~/.claude/session_telemetry_log.jsonl` |
+
+## Output Format
+
+```json
+{
+  "skill_name": "Research",
+  "analyzed_sessions": 150,
+  "co_occurring_skills": [
+    {
+      "skill_a": "Research",
+      "skill_b": "Browser",
+      "co_occurrence_count": 42,
+      "conflict_score": 0.12,
+      "avg_errors_together": 1.5,
+      "avg_errors_alone": 1.3
+    }
+  ],
+  "conflict_candidates": [
+    {
+      "skill_a": "Research",
+      "skill_b": "Content",
+      "co_occurrence_count": 15,
+      "conflict_score": 0.45,
+      "avg_errors_together": 3.2,
+      "avg_errors_alone": 1.1
+    }
+  ],
+  "generated_at": "2026-03-04T12:00:00.000Z"
+}
+```
+
+## How It Works
+
+The analyzer is a pure function that computes conflict scores from telemetry:
+
+1. Filters sessions where `skills_triggered` includes the target skill
+2. For each co-occurring skill, computes:
+   - Average errors when both skills are triggered together
+   - Average errors when each skill is triggered alone
+   - `conflict_score = clamp((errors_together - errors_alone) / (errors_alone + 1), 0, 1)`
+3. Pairs with `conflict_score > 0.3` are flagged as conflict candidates
+4. Results sorted by co-occurrence count (most common first)
+
+## Steps
+
+### 1. Run Analysis
+
+```bash
+selftune eval composability --skill Research
+```
+
+### 2. Interpret Results
+
+| Conflict Score | Interpretation |
+|---------------|---------------|
+| 0.0–0.1 | No conflict — skills work well together |
+| 0.1–0.3 | Minor friction — monitor but no action needed |
+| 0.3–0.6 | Moderate conflict — investigate trigger overlap |
+| 0.6–1.0 | Severe conflict — skills likely interfere with each other |
+
+### 3. Address Conflicts
+
+When conflict candidates are identified, present them to the user with recommended actions:
+- Check for trigger keyword overlap between the skills
+- Check if one skill's workflow interferes with the other's
+- Consider evolving descriptions to reduce false triggers
+- Use the `pattern-analyst` agent for deeper cross-skill analysis
+
+## Subagent Escalation
+
+For deep cross-skill analysis beyond what the composability command provides,
+spawn the `pattern-analyst` agent as a subagent. This is useful when conflict
+scores are high (> 0.3) and you need a full resolution plan with trigger
+ownership recommendations.
+
+## Common Patterns
+
+**"Are there conflicts between my skills?"**
+> `selftune eval composability --skill Research`
+
+**"Check composability for recent sessions only"**
+> `selftune eval composability --skill pptx --window 7`
+
+**"Which skills conflict with Research?"**
+> Run composability and check the `conflict_candidates` array.
+
+**"Why are sessions with multiple skills failing?"**
+> Run composability for each skill involved, look for high conflict scores.

package/skill/Workflows/Contribute.md

@@ -0,0 +1,94 @@
+# selftune Contribute Workflow
+
+Export anonymized skill observability data as a JSON bundle for community
+contribution. Helps improve selftune's skill routing without exposing
+private data.
+
+## When to Use
+
+- The user asks to contribute data, share usage patterns, or help improve selftune
+- The user wants to export anonymized skill observability data
+- The agent needs to submit eval data for community skill evolution
+
+## Default Command
+
+```bash
+selftune contribute --skill selftune
+```
+
+## Options
+
+| Flag | Description |
+|------|-------------|
+| `--skill <name>` | Skill to contribute data for (default: "selftune") |
+| `--output <path>` | Output file path (default: auto-generated in ~/.selftune/contributions/) |
+| `--preview` | Show what would be shared without writing |
+| `--sanitize <level>` | `conservative` (default) or `aggressive` |
+| `--since <date>` | Only include data from this date onward |
+| `--submit` | Auto-create GitHub Issue via `gh` CLI |
+
+## Sanitization Levels
+
+### Conservative (default)
+
+| Pattern | Replacement |
+|---------|-------------|
+| File paths | `[PATH]` |
+| Email addresses | `[EMAIL]` |
+| API keys, tokens, JWTs | `[SECRET]` |
+| IP addresses | `[IP]` |
+| Project name from cwd | `[PROJECT]` |
+| Session IDs | `[SESSION]` |
+
+### Aggressive
+
+Extends conservative with:
+
+| Pattern | Replacement |
+|---------|-------------|
+| camelCase/PascalCase identifiers > 8 chars | `[IDENTIFIER]` |
+| Quoted strings | `[STRING]` |
+| Import/require module paths | `[MODULE]` |
+| Queries > 200 chars | Truncated |
+
+## Bundle Contents
+
+The contribution bundle includes:
+- **Positive queries** -- queries that triggered the skill (sanitized)
+- **Eval entries** -- trigger eval set for the skill
+- **Grading summary** -- aggregate pass rates (no raw transcripts)
+- **Evolution summary** -- proposal counts and outcomes
+- **Session metrics** -- average turns, tool usage, error rates
+
+No raw transcripts, file contents, or identifiable information is included.
+
+## Submission
+
+- Default: writes JSON file to `~/.selftune/contributions/`
+- `--submit`: creates a GitHub Issue with the bundle
+  - Small bundles (< 50KB): inlined in issue body
+  - Large bundles (>= 50KB): uploaded as a gist
+
+## Steps
+
+1. Run `selftune contribute --preview --skill selftune` to preview the contribution bundle
+2. Parse the output and report the sanitized data summary to the user for review
+3. Run `selftune contribute --skill selftune` to write the bundle
+4. If the user wants to submit directly, run `selftune contribute --skill selftune --submit`
+
+## Common Patterns
+
+**User wants to see what would be shared**
+> Run `selftune contribute --preview`. Parse the output and report the
+> sanitized data summary to the user before proceeding.
+
+**User requests stronger anonymization**
+> Run `selftune contribute --sanitize aggressive`. This replaces identifiers,
+> quoted strings, and module paths in addition to standard PII scrubbing.
+
+**User wants to submit directly**
+> Run `selftune contribute --submit`. This creates a GitHub Issue via `gh`
+> CLI with the bundle inlined or uploaded as a gist.
+
+**User wants to limit to recent data**
+> Run `selftune contribute --since <date>` with the user's specified date.

package/skill/Workflows/Cron.md

@@ -0,0 +1,132 @@
+# selftune Cron Workflow
+
+Set up scheduled automation for the selftune pipeline. Auto-detects the
+platform (system cron, macOS launchd, Linux systemd) or can target
+OpenClaw-specific cron integration.
+
+## When to Use
+
+- Setting up selftune automation for the first time
+- Checking which cron jobs are registered
+- Removing selftune cron jobs (cleanup or reconfiguration)
+- Enabling the autonomous observe-grade-evolve-deploy loop
+
+## Commands
+
+### `selftune cron setup`
+
+Auto-detect the current platform and install scheduled jobs.
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--platform <name>` | Force a specific platform (`openclaw`, `cron`, `launchd`, `systemd`) | Auto-detect |
+| `--dry-run` | Preview without installing | Off |
+| `--tz <timezone>` | IANA timezone for job schedules (OpenClaw only) | Flag > `TZ` env > system timezone |
+
+Platform auto-detection: macOS → launchd, Linux → systemd, other → cron.
+
+### `selftune cron setup --platform openclaw`
+
+Register selftune cron jobs with OpenClaw. Requires OpenClaw installed and on PATH.
+
+```bash
+which openclaw    # Must resolve
+```
+
+### `selftune cron list`
+
+Show all registered selftune cron jobs. Reads from
+`~/.openclaw/cron/jobs.json` and filters for `selftune-*` entries.
+No flags.
+
+### `selftune cron remove`
+
+Remove all selftune cron jobs from OpenClaw.
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--dry-run` | Preview which jobs would be removed without deleting | Off |
+
+## Aliases
+
+`selftune schedule` is an alias for `selftune cron`. Existing `selftune schedule`
+invocations with flags (e.g. `selftune schedule --platform launchd`) continue to work.
+
+## Default Job Schedule
+
+Setup registers these jobs:
+
+| Name | Cron Expression | Schedule | Description |
+|------|----------------|----------|-------------|
+| `selftune-sync` | `*/30 * * * *` | Every 30 minutes | Sync source-truth telemetry |
+| `selftune-status` | `0 8 * * *` | Daily at 8am | Health check — report skills with pass rate below 80% |
+| `selftune-orchestrate` | `0 */6 * * *` | Every 6 hours | Full autonomous loop: sync → candidate selection → evolve → watch |
+
+All jobs run in **isolated session** mode — each execution gets a clean
+session with no context accumulation from previous runs.
+
+## Output
+
+- **setup:** Installs platform-appropriate schedule artifacts and activates them
+- **setup --platform openclaw:** Registers jobs via `openclaw cron add` and confirms each
+- **list:** Prints a formatted table of registered selftune cron jobs (name, schedule, description)
+- **remove:** Deletes each selftune cron job via `openclaw cron remove` and confirms
+
+## Steps
+
+1. Run `selftune cron setup --dry-run` to preview what would be installed
+2. Run `selftune cron setup` to install scheduled jobs for your platform
+3. Verify with `selftune status` after the first scheduled run fires
+
+For OpenClaw specifically:
+1. Run `selftune cron setup --platform openclaw --dry-run` to preview
+2. Run `selftune cron setup --platform openclaw` to register jobs
+3. Run `selftune cron list` to verify jobs are registered
+
+## The Autonomous Evolution Loop
+
+When scheduled jobs are active, selftune operates as a self-correcting system.
+The OS scheduler calls the CLI binary directly — no agent session is needed,
+no token cost for routine runs.
+
+```text
+OS scheduler fires (cron/launchd/systemd)
+    |
+    v
+selftune orchestrate --max-skills 3   (CLI runs directly, no agent)
+    |
+    v
+sync → candidate selection → evolve → validate → deploy → watch
+    |
+    v
+Improved SKILL.md written to disk
+    |
+    v
+Next interactive agent session uses updated description
+```
+
+This is distinct from interactive mode where the user says "improve my skills"
+and the agent runs orchestrate. Automated mode is for routine maintenance;
+interactive mode is for user-directed improvements.
+
+## Safety Controls
+
+| Control | How It Works |
+|---------|-------------|
+| Dry-run first | `selftune cron setup --dry-run` previews commands before installing |
+| Regression threshold | Evolution only deploys if improvement exceeds 5% on existing triggers |
+| Auto-rollback | `selftune watch` automatically rolls back if pass rate drops below baseline minus threshold |
+| Audit trail | Every evolution recorded in `evolution_audit_log.jsonl` with full history |
+| SKILL.md backup | `.bak` file created before every deploy — primary rollback path exists via .bak; fallback depends on audit metadata integrity |
+| Human override | `selftune evolve rollback --skill <name> --skill-path <path>` available anytime to manually revert |
+| Pin descriptions | Config flag to freeze specific skills and prevent evolution on sensitive skills |
+
+## Common Patterns
+
+- **User wants autonomous skill evolution** -- Run `selftune cron setup`. Auto-detects the platform and installs appropriate scheduled jobs.
+- **User specifies OpenClaw** -- Run `selftune cron setup --platform openclaw`.
+- **User wants to preview before installing** -- Run `selftune cron setup --dry-run` to show exactly what would be installed without changing anything.
+- **User needs a specific timezone (OpenClaw)** -- Run `selftune cron setup --platform openclaw --tz America/New_York`.
+- **User asks what jobs are registered** -- Run `selftune cron list`. Shows a table of all selftune cron jobs with their schedules and descriptions.
+- **User wants to remove cron automation** -- Run `selftune cron remove`. Preview first with `selftune cron remove --dry-run`.
+- **Skill regressed after cron evolution** -- The watch job should catch this automatically. If not, run `selftune evolve rollback --skill <name> --skill-path <path>` manually. See `Workflows/Rollback.md`.

package/skill/Workflows/Dashboard.md

@@ -0,0 +1,214 @@
+# selftune Dashboard Workflow
+
+Visual dashboard for selftune telemetry, skill performance, evolution
+audit, and monitoring data. Supports static HTML export, file output,
+and a live server with polling-based auto-refresh and action buttons.
+
+## Default Command
+
+```bash
+selftune dashboard
+```
+
+Opens a standalone HTML dashboard in the default browser with embedded
+data from all selftune log files.
+
+## Options
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--export` | Export data-embedded HTML to stdout | Off |
+| `--out FILE` | Write data-embedded HTML to FILE | None |
+| `--serve` | Start live dashboard server | Off |
+| `--port <port>` | Custom port for live server (requires `--serve`) | 3141 |
+
+## Modes
+
+### Static (Default)
+
+Builds an HTML file with all telemetry data embedded as JSON, saves it
+to `~/.selftune/dashboard.html`, and opens it in the default browser.
+The data is a point-in-time snapshot -- refresh by re-running the command.
+
+```bash
+selftune dashboard
+```
+
+### Export
+
+Writes the same data-embedded HTML to stdout. Useful for piping to other
+tools or capturing output programmatically.
+
+```bash
+selftune dashboard --export > dashboard.html
+```
+
+### File
+
+Writes the data-embedded HTML to a specific file path.
+
+```bash
+selftune dashboard --out /tmp/report.html
+```
+
+### Live Server
+
+Starts a Bun HTTP server with a React SPA dashboard. The SPA uses
+TanStack Query polling to auto-refresh data (overview every 15s,
+orchestrate runs every 30s, doctor every 30s) and provides action
+buttons to trigger selftune commands.
+
+```bash
+selftune dashboard --serve
+selftune dashboard --serve --port 8080
+```
+
+## Live Server
+
+### Default Port
+
+The live server binds to `localhost:3141` by default. Use `--port` to
+override.
+
+### Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/` | Serve dashboard SPA shell |
+| `GET` | `/api/v2/overview` | SQLite-backed overview payload |
+| `GET` | `/api/v2/skills/:name` | SQLite-backed per-skill report |
+| `GET` | `/api/v2/orchestrate-runs` | Recent orchestrate run reports |
+| `GET` | `/api/v2/doctor` | System health diagnostics (config, logs, hooks, evolution) |
+| `GET` | `/api/health` | Dashboard server health probe |
+| `POST` | `/api/actions/watch` | Trigger `selftune watch` for a skill |
+| `POST` | `/api/actions/evolve` | Trigger `selftune evolve` for a skill |
+| `POST` | `/api/actions/rollback` | Trigger `selftune evolve rollback` for a skill |
+
+### Auto-Refresh
+
+The dashboard SPA uses TanStack Query with `refetchInterval` to poll
+the v2 API endpoints automatically:
+
+- `/api/v2/overview` — every 15 seconds
+- `/api/v2/orchestrate-runs` — every 30 seconds
+- `/api/v2/doctor` — every 30 seconds
+- `/api/v2/skills/:name` — every 30 seconds (when viewing a skill)
+
+Data also refreshes on window focus. No SSE or websocket connection
+is required.
+
+### Action Endpoints
+
+Action buttons in the dashboard trigger selftune commands via POST
+requests. Each endpoint spawns a `bun run` subprocess.
+
+**Watch and Evolve** request body:
+
+```json
+{
+  "skill": "skill-name",
+  "skillPath": "/path/to/SKILL.md"
+}
+```
+
+**Rollback** request body:
+
+```json
+{
+  "skill": "skill-name",
+  "skillPath": "/path/to/SKILL.md",
+  "proposalId": "proposal-uuid"
+}
+```
+
+All action endpoints return:
+
+```json
+{
+  "success": true,
+  "output": "command stdout",
+  "error": null
+}
+```
+
+On failure, `success` is `false` and `error` contains the error message.
+
+### Browser and Shutdown
+
+The live server auto-opens the dashboard URL in the default browser on
+macOS (`open`) and Linux (`xdg-open`).
+
+Graceful shutdown on `SIGINT` (Ctrl+C) and `SIGTERM`: closes the SQLite
+database and stops the server.
+
+## Data Contents
+
+The dashboard displays data from these sources:
+
+| Data | Source | Description |
+|------|--------|-------------|
+| Telemetry | `session_telemetry_log.jsonl` | Session-level telemetry records |
+| Skills | `skill_usage_log.jsonl` | Skill activation and usage events |
+| Queries | `all_queries_log.jsonl` | All user queries across sessions |
+| Evolution | `evolution_audit_log.jsonl` | Evolution audit trail (create, deploy, rollback) |
+| Decisions | `~/.selftune/memory/` | Evolution decision records |
+| Snapshots | Computed | Per-skill monitoring snapshots (pass rate, regression status) |
+| Unmatched | Computed | Queries that did not trigger any skill |
+| Pending | Computed | Evolution proposals not yet deployed, rejected, or rolled back |
+
+If no log data is found, the static modes exit with an error message
+listing the checked file paths.
+
+## Steps
+
+### 1. Choose Mode
+
+| Goal | Command |
+|------|---------|
+| Quick visual check | `selftune dashboard` |
+| Save report to file | `selftune dashboard --out report.html` |
+| Pipe to another tool | `selftune dashboard --export` |
+| Live monitoring | `selftune dashboard --serve` |
+
+### 2. Run Command
+
+```bash
+# Static (opens browser)
+selftune dashboard
+
+# Live server
+selftune dashboard --serve
+```
+
+### 3. Interact with Dashboard
+
+- **Static mode**: View the snapshot. Re-run to refresh.
+- **Live mode**: Data refreshes automatically via polling (15-30s intervals).
+  Use action buttons to trigger watch, evolve, or rollback directly from
+  the dashboard.
+
+## Common Patterns
+
+**User wants to see skill performance visually**
+> Run `selftune dashboard`. This opens a browser with a point-in-time snapshot.
+> Report to the user that the dashboard is open.
+
+**User wants live monitoring**
+> Run `selftune dashboard --serve`. Inform the user that data refreshes
+> automatically every 15-30 seconds via polling.
+
+**User wants a shareable report**
+> Run `selftune dashboard --out report.html`. Report the file path to the
+> user. The HTML file is self-contained with all data embedded.
+
+**Dashboard shows no data**
+> Run `selftune doctor` to verify hooks are installed. If hooks are missing,
+> route to the Initialize workflow. If hooks are present but no sessions
+> have run, inform the user that sessions must generate telemetry first.
+
+**User wants a different port**
+> Run `selftune dashboard --serve --port <port>`. Port must be 1-65535.
+
+**User wants to trigger actions from the dashboard**
+> Run `selftune dashboard --serve` for live mode. The dashboard provides
+> action buttons for watch, evolve, and rollback per skill via POST endpoints.