@jaggerxtrm/specialists 3.12.0 → 3.14.0

package/config/skills/using-kpi/SKILL.md

@@ -143,6 +143,92 @@ sp db stats --with-payload --format json \
         end'
 ```
 
+## Recipe 7 — payload component breakdown per specialist
+
+**Truth source first.** The actual prompt size billed by the API is the first turn's `input_tokens` from `token_trajectory_json[0]`. Use it as the ground truth — `payload_breakdown` events undercount (tool definitions and harness framing are not captured) and historical rows before the rule N× fix overcount mandatory_rule by attached-rule count.
+
+```bash
+DB=.specialists/db/observability.db
+sqlite3 "$DB" "SELECT specialist, model, AVG(json_extract(token_trajectory_json, '\$[0].token_usage.input_tokens')) AS avg_first_in, COUNT(*) AS n FROM specialist_job_metrics WHERE token_trajectory_json IS NOT NULL AND status='done' GROUP BY specialist, model ORDER BY avg_first_in DESC"
+```
+
+Use this number for cost decisions. Use `payload_breakdown` only for *relative* component analysis (which knob to tune), not absolute sizing.
+
+`sp db stats --with-payload` only surfaces total `payload_kb` / `payload_tokens`. To audit *what* fills the prompt (system_prompt vs mandatory rules vs skills vs bead_context vs memory), query `payload_breakdown` events directly. Use this for eager-load bloat investigations, prompt/rule consolidation planning, or duplication hunts — but cross-check against the truth source above.
+
+```bash
+DB=.specialists/db/observability.db
+sqlite3 "$DB" "SELECT specialist, event_json FROM specialist_events WHERE type='payload_breakdown' GROUP BY specialist ORDER BY t DESC" \
+  | python3 -c '
+import json, sys
+rows = []
+for line in sys.stdin:
+    if "|" not in line: continue
+    spec, js = line.split("|", 1)
+    d = json.loads(js)
+    agg = {}
+    for c in d["payload_breakdown"]["components"]:
+        a = agg.setdefault(c["kind"], {"tokens":0,"n":0})
+        a["tokens"] += c["tokens"]; a["n"] += 1
+    rows.append((spec, d["payload_breakdown"]["totals"]["tokens"], agg))
+rows.sort(key=lambda r: -r[1])
+print(f"{\"specialist\":<22}{\"total\":>8}{\"rules\":>8}{\"rules_n\":>8}{\"sys\":>8}{\"skills\":>8}{\"bead\":>8}{\"mem\":>8}")
+for s, t, a in rows:
+    g = lambda k: a.get(k, {"tokens":0,"n":0})
+    print(f"{s:<22}{t:>8}{g(\"mandatory_rule\")[\"tokens\"]:>8}{g(\"mandatory_rule\")[\"n\"]:>8}{g(\"system_prompt\")[\"tokens\"]:>8}{g(\"skill\")[\"tokens\"]:>8}{g(\"bead_context\")[\"tokens\"]:>8}{g(\"memory\")[\"tokens\"]:>8}")
+'
+```
+
+Component kinds: `system_prompt`, `mandatory_rule` (one event entry per attached rule), `skill` (path/description label only — full bodies are eagerly injected at runtime but NOT counted here), `task_template`, `bead_context`, `memory`.
+
+**Important:** `skill` entries in `payload_breakdown` show only the path/description label (~10-40 tokens). The full skill body is forcefully injected via `skills.paths` on every run and IS billed as input tokens. To measure the real eager-skill cost, see Recipe 8.
+
+Optimization signals (from breakdown alone):
+- `mandatory_rule` total dominates: audit wrapper inflation by comparing `bytes` per rule in the event vs `wc -c config/mandatory-rules/<id>.md`. Mismatch >5x means a wrapper or richer source is adding hidden cost.
+- `bead_context` huge: bead description is bloated — orchestrator should write more concise contracts.
+- `memory` huge: stale or noisy memories — run `bd memories` cleanup or consolidation.
+
+## Recipe 8 — eager skill-body cost per specialist
+
+`skills.paths` are eagerly injected on every run; the bodies appear in the API-billed prompt but the `payload_breakdown` event records only the path label. To derive the real eager-skill cost:
+
+```
+eager_skill_cost ≈ first_turn_input_tokens − sum(payload_breakdown non-skill components)
+                   − constant per-specialist framing/tool-defs overhead
+```
+
+Two-step audit:
+
+```bash
+# Step 1: real first-turn input tokens per specialist (truth)
+DB=.specialists/db/observability.db
+sqlite3 "$DB" "
+  SELECT specialist, AVG(json_extract(token_trajectory_json, '\$[0].token_usage.input_tokens')) AS avg_first_in, COUNT(*) AS n
+  FROM specialist_job_metrics
+  WHERE token_trajectory_json IS NOT NULL AND status='done'
+  GROUP BY specialist ORDER BY avg_first_in DESC"
+
+# Step 2: per-specialist measured non-skill components (post-kdl4n)
+sqlite3 "$DB" "SELECT specialist, event_json FROM specialist_events WHERE type='payload_breakdown' GROUP BY specialist ORDER BY t DESC" \
+  | python3 -c '
+import json, sys
+for line in sys.stdin:
+    if "|" not in line: continue
+    spec, js = line.split("|", 1)
+    d = json.loads(js)
+    non_skill = sum(c["tokens"] for c in d["payload_breakdown"]["components"] if c["kind"] != "skill")
+    print(f"{spec:<22}{non_skill:>10}")
+'
+```
+
+Then `delta = first_in − non_skill_total`. The framing/tool-defs constant is roughly the same across specialists with the same model — you can estimate it by running a specialist with NO `skills.paths` attached as a baseline.
+
+Per-skill body weight: `wc -c <skill-path>/SKILL.md` divided by 4 (cl100k_base approximation). High-frequency, large-body skills are the inlining candidates; low-frequency or small ones stay attached.
+
+Optimization signals (skills):
+- `delta` >> sum of attached skill body bytes/4: framing/tool defs are the bulk — leave skills alone.
+- `delta` ≈ sum of skill body weights: skills dominate eager cost — inline frequently-used hot guidance into `system_prompt`, keep rare deep references as skills, consider splitting big mixed skills.
+
 ## References
 
 - `docs/observability-metrics.md`

package/config/skills/using-script-specialists/SKILL.md

@@ -9,7 +9,9 @@ description: >
   output immediately, or when the work is a single LLM call with structured
   input/output. Do NOT use for tracked agent work — that belongs to
   `using-specialists-v2`.
-version: 1.0
+version: 1.1.0
+updated: 2026-05-06
+synced_at: a0e54d0c
 ---
 
 # Script-Class Specialists
@@ -54,7 +56,7 @@ A spec is rejected at request time (`specialist_load_error`) if any of:
 - `execution.interactive` is `true`
 - `execution.requires_worktree` is `true`
 - `execution.permission_required` is anything other than `READ_ONLY`
-- `skills.scripts` is non-empty
+- `skills.scripts` is non-empty (always rejected; no `--allow-local-scripts` bypass)
 - `prompt.task_template` is missing
 - a referenced `$var` in the chosen template is not supplied (`template_variable_missing`)
 
@@ -101,9 +103,9 @@ sp script <specialist-name> \
 Behaviour:
 
 - Loads the spec via `SpecialistLoader` (same loader as `sp run`).
-- Renders `prompt.task_template` (or named template) with `--vars`.
-- Spawns `pi --mode json --no-session --no-extensions --no-tools` with the
-  resolved model.
+- Renders `prompt.task_template` (or named template) with `--vars`, then feeds the rendered prompt via stdin.
+- `--db-path /path/to/observability.db` is an exact SQLite file path; omit it to use the project default `.specialists/db/observability.db`.
+- Spawns `pi` in JSON mode with no session, no extensions, no tools, and offline; forwards the resolved model, optional `--thinking`, and `--system-prompt` when `prompt.system` is set (full override, not append).
 - Returns the final assistant text on stdout. With `--json`, returns the full
   `ScriptGenerateResult` envelope.
 - Writes one row to `.specialists/db/observability.db` (same writer as `sp run`).

package/config/skills/using-specialists-v2/SKILL.md

@@ -272,7 +272,7 @@ sp epic abandon <epic-id> --reason "..."
 sp end
 ```
 
-`sp result <job-id>` returns the most recent completed turn for `waiting` jobs with a `Session is waiting for your input` footer — use it to inspect a keep-alive job before deciding whether to resume. For `running` jobs, `sp feed <job-id>` is the right tool; `sp poll` is deprecated. Avoid `specialists status --job` for normal monitoring; prefer `sp ps <job-id>`.
+`sp result <job-id>` returns the most recent completed turn for `waiting` jobs with a `Session is waiting for your input` footer — use it to inspect a keep-alive job before deciding whether to resume. For `running` jobs, `sp feed <job-id>` is the right tool. Avoid `specialists status --job` for normal monitoring; prefer `sp ps <job-id>`.
 
 ## Flag Semantics