@jaggerxtrm/specialists 3.10.0 → 3.12.0

package/config/skills/using-kpi/SKILL.md

@@ -0,0 +1,150 @@
+---
+name: using-kpi
+description: >-
+  Analyze specialist KPI data in observability SQLite. Use for runtime, payload,
+  waiting, tool-call, and outlier analysis. Token estimates use cl100k_base-style
+  approximation with ~±5% accuracy.
+gemini-command: using-kpi
+version: 3.1.0
+---
+
+# using-kpi
+
+KPI analysis skill for `sp db stats` / `sp db extract` data.
+
+## Quick rule
+
+`active_runtime_ms` = real paid runtime. Rank by that first. `elapsed_ms` is total wall time. `waiting_ms` catches forgotten keep-alives.
+
+Token counts are approximate, cl100k_base-style, about ±5%. Bytes are exact UTF-8 size.
+
+## Recipe 1 — specialist × model leaderboard by active cost
+
+```bash
+sp db stats --format json \
+  | jq -r '
+      .rows
+      | group_by([.specialist, .model])
+      | map({
+          specialist: .[0].specialist,
+          model: .[0].model,
+          jobs: length,
+          active_ms: (map((.active_runtime_ms // 0)) | add),
+          total_ms: (map((.total_runtime_ms // .elapsed_ms // 0)) | add),
+          turns: (map((.total_turns // 0)) | add),
+          tools: (map((.total_tools // 0)) | add),
+          payload_kb: (map((.payload_kb // 0)) | add)
+        })
+      | sort_by(-.active_ms, -.jobs)
+      | .[]
+      | [ .specialist, .model, .jobs, .active_ms, .total_ms, .turns, .tools, .payload_kb ]
+      | @tsv'
+```
+
+## Recipe 2 — outliers above p95
+
+```bash
+sp db stats --format json \
+  | jq '
+      .rows as $rows
+      | {
+          active: ($rows | map(.active_runtime_ms // 0) | sort),
+          tools: ($rows | map(.total_tools // 0) | sort),
+          turns: ($rows | map(.total_turns // 0) | sort),
+          payload: ($rows | map(.payload_kb // 0) | sort)
+        } as $s
+      | {
+          active_p95: $s.active[(($s.active|length)*95/100|floor)],
+          tools_p95: $s.tools[(($s.tools|length)*95/100|floor)],
+          turns_p95: $s.turns[(($s.turns|length)*95/100|floor)],
+          payload_p95: $s.payload[(($s.payload|length)*95/100|floor)]
+        } as $p
+      | $rows
+      | map(select(
+          ((.active_runtime_ms // 0) >= $p.active_p95) or
+          ((.total_tools // 0) >= $p.tools_p95) or
+          ((.total_turns // 0) >= $p.turns_p95) or
+          ((.payload_kb // 0) >= $p.payload_p95)
+        ))
+      | .[]
+      | [ .job_id, .specialist, .model, .active_runtime_ms, .total_tools, .total_turns, .payload_kb ]
+      | @tsv'
+```
+
+## Recipe 3 — payload bloat ranking
+
+```bash
+sp db stats --with-payload --format json \
+  | jq -r '
+      .rows
+      | group_by(.specialist)
+      | map({
+          specialist: .[0].specialist,
+          jobs: length,
+          avg_payload_kb: ((map((.payload_kb // 0)) | add) / length),
+          max_payload_kb: (map((.payload_kb // 0)) | max)
+        })
+      | sort_by(-.avg_payload_kb)
+      | .[:10]
+      | .[]
+      | [ .specialist, .jobs, (.avg_payload_kb|tostring), (.max_payload_kb|tostring) ]
+      | @tsv'
+```
+
+## Recipe 4 — waiting-state hygiene
+
+```bash
+sp db stats --format json \
+  | jq -r '
+      .rows
+      | map(select((.waiting_s? // 0) != 0))
+      | map(. + {waiting_ratio: ((.waiting_ms // 0) / ((.total_runtime_ms // .elapsed_ms // 1) + 0.0))})
+      | sort_by(-.waiting_ratio, -.waiting_ms)
+      | .[]
+      | [ .job_id, .specialist, .model, (.waiting_ms|tostring), (.total_runtime_ms // .elapsed_ms|tostring), (.waiting_ratio|tostring) ]
+      | @tsv'
+```
+
+## Recipe 5 — tool-call distribution per specialist
+
+```bash
+sp db stats --format json \
+  | jq -r '
+      .rows
+      | group_by(.specialist)
+      | map({
+          specialist: .[0].specialist,
+          counts: (map(.tool_call_counts_json? // "{}")
+            | map(fromjson)
+            | add)
+        })
+      | .[]
+      | .counts
+      | to_entries
+      | sort_by(-.value)
+      | .[]
+      | [ .key, .value ]
+      | @tsv'
+```
+
+## Recipe 6 — payload vs active runtime correlation
+
+```bash
+sp db stats --with-payload --format json \
+  | jq -r '
+      .rows
+      | map(select((.payload_kb? // 0) > 0 and ((.active_runtime_ms? // 0) > 0)))
+      | map([(.payload_kb|tonumber), (.active_runtime_ms|tonumber)])
+      | if length < 2 then empty else
+          (map(.[0]) | add / length) as $mx |
+          (map(.[1]) | add / length) as $my |
+          (map((.[0]-$mx)*(.[1]-$my)) | add) /
+          ((map((.[0]-$mx)^2) | add) * (map((.[1]-$my)^2) | add)) ^ 0.5
+        end'
+```
+
+## References
+
+- `docs/observability-metrics.md`
+- `src/cli/db.ts`
+- `src/specialist/observability-sqlite.ts`

package/config/skills/using-script-specialists/SKILL.md

@@ -0,0 +1,208 @@
+---
+name: using-script-specialists
+description: >
+  Use this skill for synchronous one-shot specialist invocations via `sp script`
+  (CLI) or `sp serve` (HTTP daemon). These run READ_ONLY, template-driven
+  specialists with `$var` substitution and return JSON in-process — no beads,
+  no chains, no worktrees, no job lifecycle. Trigger when integrating a
+  specialist into a service, script, or library, when the caller needs the
+  output immediately, or when the work is a single LLM call with structured
+  input/output. Do NOT use for tracked agent work — that belongs to
+  `using-specialists-v2`.
+version: 1.0
+---
+
+# Script-Class Specialists
+
+`sp script` and `sp serve` are a separate runtime from the bead-first
+orchestration covered by `using-specialists-v2`. They exist for service and
+library integration, not for agent chains.
+
+| Aspect | `sp run` (orchestration) | `sp script` / `sp serve` |
+| --- | --- | --- |
+| Driver | bead contract | template + variables |
+| Execution | supervised job, async | one-shot, synchronous |
+| Permissions | READ_ONLY / MEDIUM / HIGH | READ_ONLY only |
+| Worktrees | edit-capable provisions one | rejected |
+| Output | result.txt + events.jsonl + bead notes | stdout JSON / HTTP body |
+| Audit | `.specialists/jobs/<id>/` | one row in `.specialists/db/observability.db` |
+
+Use `sp script` from a shell or build pipeline. Use `sp serve` from a service
+that needs an HTTP endpoint backed by `pi`. The same `.specialist.json` runs
+under both.
+
+## When To Use This Skill
+
+Trigger when:
+
+- A service or script needs a single LLM-backed transform (summarize, classify,
+  extract) returning JSON.
+- You are integrating specialists into Python/Node code that cannot block on a
+  supervised job lifecycle.
+- The call is request/response shaped: variables in, structured output out.
+- You need a sidecar HTTP endpoint (`sp serve`) to wrap a specialist for a
+  service consumer that already speaks HTTP.
+
+Do NOT trigger for: code review, debugging, implementation, multi-turn work,
+keep-alive sessions, anything that should write files. Those belong to
+`using-specialists-v2`.
+
+## Specialist Compatibility (compatGuard)
+
+A spec is rejected at request time (`specialist_load_error`) if any of:
+
+- `execution.interactive` is `true`
+- `execution.requires_worktree` is `true`
+- `execution.permission_required` is anything other than `READ_ONLY`
+- `skills.scripts` is non-empty
+- `prompt.task_template` is missing
+- a referenced `$var` in the chosen template is not supplied (`template_variable_missing`)
+
+Author specs that explicitly target script-class:
+
+```json
+{
+  "specialist": {
+    "metadata": { "name": "summarize-event", "version": "1.0.0", "category": "ingestion" },
+    "execution": {
+      "mode": "auto",
+      "model": "anthropic/claude-haiku-4-5",
+      "timeout_ms": 30000,
+      "interactive": false,
+      "response_format": "json",
+      "output_type": "custom",
+      "permission_required": "READ_ONLY",
+      "requires_worktree": false,
+      "max_retries": 0
+    },
+    "prompt": {
+      "task_template": "Summarize event $event_id with body: $body. Return JSON {\"summary\": \"...\"}.",
+      "output_schema": { "required": ["summary"] }
+    }
+  }
+}
+```
+
+## `sp script` — One-Shot CLI
+
+```bash
+sp script <specialist-name> \
+  --vars key1=value1 --vars key2=value2 \
+  [--template task_template] \
+  [--model anthropic/claude-sonnet-4-6] \
+  [--thinking medium] \
+  [--timeout-ms 60000] \
+  [--db-path /path/to/observability.db] \
+  [--single-instance <lock-name>] \
+  [--no-trace] \
+  [--json]
+```
+
+Behaviour:
+
+- Loads the spec via `SpecialistLoader` (same loader as `sp run`).
+- Renders `prompt.task_template` (or named template) with `--vars`.
+- Spawns `pi --mode json --no-session --no-extensions --no-tools` with the
+  resolved model.
+- Returns the final assistant text on stdout. With `--json`, returns the full
+  `ScriptGenerateResult` envelope.
+- Writes one row to `.specialists/db/observability.db` (same writer as `sp run`).
+
+Exit codes:
+
+- `0` — success.
+- non-zero — failure; with `--json`, body has `success: false` and `error_type`.
+
+Use `--single-instance <lock>` when concurrent invocations of the same logical
+job must be serialized (cron, batch script).
+
+## `sp serve` — HTTP Daemon
+
+```bash
+sp serve \
+  [--port 8000] \
+  [--concurrency 4] \
+  [--queue-timeout-ms 5000] \
+  [--shutdown-grace-ms 30000] \
+  [--project-dir /path/to/project] \
+  [--fallback-model anthropic/claude-haiku-4-5]
+```
+
+POST `/v1/generate`:
+
+```json
+{
+  "specialist": "summarize-event",
+  "variables": { "event_id": "abc", "body": "..." },
+  "template": "task_template",
+  "model_override": "anthropic/...",
+  "timeout_ms": 60000,
+  "trace": true
+}
+```
+
+Response (200, success):
+
+```json
+{
+  "success": true,
+  "output": "<final text>",
+  "parsed_json": { "summary": "..." },
+  "meta": {
+    "specialist": "summarize-event",
+    "model": "anthropic/claude-haiku-4-5",
+    "duration_ms": 1234,
+    "trace_id": "<uuid>"
+  }
+}
+```
+
+Response (200, failure):
+
+```json
+{ "success": false, "error": "...", "error_type": "..." }
+```
+
+Error types: `specialist_not_found | specialist_load_error |
+template_variable_missing | auth | quota | timeout | network | invalid_json |
+output_too_large | internal`.
+
+`400` is reserved for malformed HTTP. `429` returns when concurrency cap is
+saturated past `queue-timeout-ms`.
+
+## Operational Rules
+
+- One `pi` subprocess per in-flight request, bounded by `--concurrency`.
+- Credentials come from `pi`'s own `~/.pi/agent/auth.json`. The service never
+  touches API keys.
+- Observability DB is shared with `sp run`. Audit trail is unified.
+- The service is sidecar-per-consumer: no multi-tenant routing, no session
+  state, no orchestration. If you need orchestration, use `sp run` + beads.
+- For container deployments, see `docs/specialists-service-install.md`. Image
+  runs as non-root UID 10001; bind-mount `~/.pi` and `.specialists/`.
+
+## When To Switch Back To `using-specialists-v2`
+
+If any of these become true mid-design, drop script-class and use the
+orchestration runtime:
+
+- The work needs to write files.
+- The caller wants a multi-turn / keep-alive session.
+- A reviewer pass is needed.
+- The work should be tracked as a bead with auditability beyond a single
+  observability row.
+- The output is iterative (steer / resume).
+
+## What Not To Put Here
+
+- Bead workflow, chains, epics, reviewers, worktrees — those live in
+  `using-specialists-v2`.
+- Orchestration MCP tooling (`use_specialist`).
+- Long-running multi-turn examples.
+
+## Reference
+
+- `docs/specialists-service.md` — HTTP contract and operational notes.
+- `docs/specialists-service-install.md` — Docker/Podman install path.
+- `docs/script-specialists.md` — historical context for the script-class shape.
+- `src/cli/script.ts`, `src/cli/serve.ts`, `src/specialist/script-runner.ts` — runtime.