@jaggerxtrm/specialists 3.16.0 → 3.18.0

package/docs/background-jobs.md

@@ -0,0 +1,203 @@
+---
+title: Background Jobs
+scope: background-jobs
+category: guide
+version: 1.8.0
+updated: 2026-06-23
+synced_at: bf6baf7a
+description: Supervisor-backed job model, keep-alive semantics, and monitoring commands.
+source_of_truth_for:
+  - "src/cli/run.ts"
+  - "src/cli/feed.ts"
+  - "src/cli/chat.ts"
+  - "src/cli/chat/control.ts"
+  - "src/cli/chat/feed.ts"
+  - "src/cli/chat/status.ts"
+  - "src/cli/steer.ts"
+  - "src/cli/resume.ts"
+  - "src/specialist/supervisor.ts"
+domain:
+  - jobs
+---
+
+# Background Jobs
+
+> `sp` is an alias for `specialists`.
+
+Every `specialists run` is DB-backed in normal runtime. `.specialists/jobs/<job-id>/` remains a legacy/operator mirror for recovery and debugging.
+
+## Start a run
+
+```bash
+specialists run sync-docs --bead unitAI-26s
+# stderr: [job started: 49adda]
+```
+
+`specialists run --background` spawns a detached child process that runs the full Supervisor-backed flow in its own process group. The parent prints the job id and exits immediately.
+
+When `tmux` is installed:
+- a named tmux session is created as `sp-<specialist>-<id>`
+- use `specialists attach <job-id>` to attach directly to that legacy tmux session
+- use `specialists list --live` for an interactive tmux session picker
+
+For a first-class interactive TUI while launching a new specialist, use `sp chat <specialist> ...`. Chat is separate from legacy tmux attach: it owns the terminal, renders the normalized feed, shows status, and sends steer/resume messages from one prompt.
+
+When `tmux` is not installed, the CLI falls back to detached process mode (stdio ignored, spawned with `detached: true`) and still keeps DB state canonical; file mirrors are legacy/operator-only.
+
+Latest job id is surfaced by active-mode detection. Legacy file mirror, when enabled, may still write:
+
+```text
+.specialists/jobs/latest
+```
+
+## Keep-alive sessions
+
+```bash
+specialists run debugger --bead unitAI-abc --keep-alive
+```
+
+You can also make keep-alive the default in specialist YAML:
+
+```yaml
+specialist:
+  execution:
+    interactive: true
+```
+
+After the first turn, keep-alive jobs transition to `waiting` and preserve full conversation context for future turns.
+
+Keep-alive sessions support automatic cleanup via `waiting_auto_close_ms` in stall detection configuration. When configured (value > 0), waiting sessions automatically close after the specified silence threshold. The close attempt has a 5-second graceful timeout; if the session close hangs, forced termination is triggered.
+
+Run-time precedence:
+- `--no-keep-alive` / `no_keep_alive` forces one-shot mode
+- `--keep-alive` / `keep_alive` forces keep-alive mode
+- otherwise `execution.interactive` decides (default `false`)
+
+## Observe progress
+
+```bash
+sp ps                         # actionable dashboard
+sp ps -f                      # TTY in-place dashboard; pipes append ANSI-free snapshots
+sp feed 49adda --follow       # per-job follow, including future resume turns
+sp feed -f                    # global follow; keep-alive waiting jobs are terminal-equivalent unless --forever
+sp feed 49adda                # compact DB-backed event replay for that job
+sp log 49adda                 # lean runtime/control/error log; parent dirs aggregate child repo DBs
+sp log --specialist reviewer -f  # follow reviewer runtime logs
+sp ps 49adda --json
+```
+
+Snapshot `sp feed <job-id>` reads compact DB event history for that job. Use `sp log <job-id>` for the lean runtime/debug stream: dispatch, control signals, resume/steer consumption, stop/SIGTERM/SIGKILL, crashes, status transitions, and terminal error provenance. It intentionally excludes feed-owned agent internals by default; add `--all-events` when you need them. Legacy file mirrors, when enabled, are recovery surfaces rather than the normal source of truth.
+
+## Interactive chat TUI
+
+Use `sp chat` when you want to launch a specialist and keep a live, interactive control surface open:
+
+```bash
+sp chat explorer "map this failure"
+sp chat reviewer --bead unitAI-929wj
+```
+
+The chat feed intentionally matches `sp feed -f` formatting. It reads timeline events from SQLite first and falls back to `events.jsonl` only for recovery. It suppresses raw launch preamble/stderr noise, shows startup/payload context side-lines, dedupes repeated streaming `THINK`/`TEXT` events once per turn, and appends the final `run_complete.output` so the result is visible in the same screen.
+
+Typed chat input uses the same FIFO as the explicit commands:
+
+| Current status | Chat input becomes | CLI equivalent |
+|---|---|---|
+| `running` / `starting` | `{"type":"steer","message":"..."}` | `sp steer <job-id> "..."` |
+| `waiting` | `{"type":"resume","task":"..."}` | `sp resume <job-id> "..."` |
+
+Slash commands in chat:
+
+- `/quit` detaches without killing the job.
+- `Ctrl+C` behaves like `/quit`: it restores the terminal and detaches, but does not stop the job.
+- `/stop` stops the job through the normal control path.
+- `/finalize` finalizes a waiting chain/job.
+- `/notes <text>` appends to the current bead.
+- `/show` prints current context.
+
+`sp attach <job-id>` is still the legacy tmux attach path. TUI attach to an already-existing job is planned separately in bead `unitAI-hx4ln`; until then, use `sp feed` + `sp steer`/`sp resume` for existing jobs that were not started by `sp chat`.
+
+## Read final output
+
+```bash
+specialists result 49adda
+```
+
+## Steer a running job
+
+`steer` works for **any running job** (keep-alive or not). It injects a mid-turn instruction and does not cancel the run.
+
+```bash
+specialists steer 49adda "focus only on supervisor.ts"
+specialists steer 49adda "skip tests and isolate root cause"
+```
+
+FIFO payload:
+
+```json
+{"type":"steer","message":"..."}
+```
+
+## Resume a waiting keep-alive job
+
+`resume` is for keep-alive sessions in `waiting` state only.
+
+```bash
+specialists resume 49adda "now write the fix"
+specialists resume 49adda "add regression tests"
+```
+
+If status is `running`, use `steer` instead.
+
+`specialists follow-up` remains as a deprecated alias that delegates to `resume`.
+
+## Stop a job
+
+```bash
+specialists stop 49adda
+```
+
+## Job files
+
+```text
+.specialists/jobs/<job-id>/
+```
+
+| File | Purpose |
+|---|---|
+| `status.json` | legacy/operator mirror of current state (`starting/running/waiting/done/error`), pid, model, bead_id, tmux_session? |
+| `events.jsonl` | legacy/operator mirror of append-only normalized timeline |
+| `result.txt` | legacy/operator mirror of final assistant output |
+| `steer.pipe` | FIFO for `steer` / `resume` messages (removed on completion) |
+
+Ready markers:
+
+```text
+.specialists/ready/
+```
+
+## Stall detection thresholds
+
+Background jobs support configurable stall detection via `specialist.stall_detection` in specialist YAML:
+
+| Option | Default | Description |
+|---|---|---|
+| `running_silence_warn_ms` | 60000 | Warn when no output received during running state |
+| `running_silence_error_ms` | 300000 | Error/cancel when running silence exceeds threshold |
+| `waiting_stale_ms` | 3600000 | Warn when waiting job has no activity |
+| `waiting_auto_close_ms` | 0 | Auto-close waiting sessions after N ms (0 = disabled) |
+| `tool_duration_warn_ms` | 120000 | Warn when tool call exceeds duration |
+
+YAML example:
+
+```yaml
+specialist:
+  stall_detection:
+    waiting_auto_close_ms: 1800000  # Auto-close after 30 min idle
+```
+
+## See also
+
+- [workflow.md](workflow.md)
+- [cli-reference.md](cli-reference.md)
+- [mcp-tools.md](mcp-tools.md)

package/docs/bare-specialists.md

@@ -0,0 +1,83 @@
+---
+title: Bare Specialists
+scope: authoring
+category: guide
+version: 1.0.0
+updated: 2026-05-23
+synced_at: bf6baf7a
+description: How to author bare-mode specialists for non-coding LLM transforms.
+source_of_truth_for:
+  - "config/specialists/bare.specialist.json"
+  - "src/specialist/runner.ts"
+  - "src/specialist/schema.ts"
+  - "config/skills/specialists-creator/SKILL.md"
+domain:
+  - authoring
+---
+
+# Bare Specialists
+
+## What bare mode is
+
+Bare mode runs specialist prompt with runtime injections stripped away, so output starts from only `prompt.system` plus `prompt.task_template` and does not pick up package-class specialist framing.
+
+## When to use it
+
+Use bare mode for non-coding LLM transforms:
+- research
+- summarization
+- extraction
+- document analysis
+- translation
+
+Do not use it for coding agents, implementation work, or specialist roles that need runtime rules, tools, or workflow framing.
+
+## What gets disabled
+
+Bare mode disables these package-runner injection sites:
+
+| Injection site | Disabled in bare mode |
+|---|---|
+| Specialist Run Context | yes |
+| Output Style | yes |
+| GitNexus mandate | yes |
+| `STATIC_WORKFLOW_RULES_BLOCK` | yes |
+| memory injection | yes |
+| GitNexus pre-query snapshot | yes |
+| reviewer patch retrieval | yes |
+| output contract | yes |
+| task-side mandatory rules | yes |
+| reviewer diff context | yes |
+
+## Orthogonality with `system_prompt_mode`
+
+| `execution.bare` | `prompt.system_prompt_mode` | Result |
+|---|---|---|
+| `false` | `append` | default package-class runtime; base prompt plus specialist runtime injections |
+| `false` | `replace` | package-class runtime with coding-agent base prompt removed; teach all behavior explicitly |
+| `true` | `append` | bare runtime; only `prompt.system` plus `prompt.task_template` matter |
+| `true` | `replace` | bare runtime; same stripped surface, with base prompt removed too |
+
+## How to create one
+
+Copy starter from installed npm package, not repo clone:
+
+```bash
+cp "$(node -p \"require.resolve('@jaggerxtrm/specialists/package.json').replace(/package\\.json$/, '')\")config/specialists/bare.specialist.json" ".specialists/user/<your-name>.specialist.json"
+```
+
+Then edit fields:
+- `metadata.name` — kebab-case specialist id
+- `metadata.description` — routing summary for `specialists list`
+- `prompt.system` — task-specific instruction set; include every behavior bare mode will not inject
+
+## Verification
+
+- `specialists list` shows your specialist
+- `sp config show <name> --resolved` shows resolved tools and runtime surface
+- `bun config/skills/specialists-creator/scripts/validate-specialist.ts <path>` validates schema
+
+## Caveats
+
+- Bare mode bypasses `mandatory_rules` entirely; put needed rules directly in `prompt.system` text instead.
+- `script-class` specialists are an alternate path for the simplest cases; see [Script-Class vs Package-Class Runtime](authoring.md#script-class-vs-package-class-runtime).

package/docs/benchmarks/executor-benchmark-runner.md

@@ -0,0 +1,66 @@
+# Executor benchmark runner (unitAI-gc2a.3)
+
+## One-command launch
+
+```bash
+node scripts/run-executor-benchmark.mjs --run-id unitAI-gc2a-r1
+```
+
+Default matrix source: `config/benchmarks/executor-benchmark-matrix.json`.
+
+## Deterministic matrix order
+
+Runner expands queue in fixed order:
+1. Task order from `tasks[]`
+2. Model order from `models[]`
+3. Replicate fixed at `1`
+
+Current matrix size: `3 tasks × 4 models × 1 rep = 12 runs`.
+
+## Isolation strategy (no cross-run contamination)
+
+Per sample runner creates fresh benchmark bead cloned from seed bead content.
+That bead gets unique title with run id + model id.
+Executor always runs with `--worktree` on that benchmark bead.
+Reviewer always runs with `--job <executor-job-id>` on same benchmark bead.
+
+Guardrail before reviewer dispatch:
+- runner checks worktree has commit diff vs `origin/main`
+- if no diff / stale state, reviewer skipped, sample marked failed (`stale_or_empty_diff`)
+
+## Artifacts
+
+Output root: `.specialists/benchmarks/runs/<run-id>/`
+
+- `manifest.json` — immutable sample plan for run id
+- `attempts.jsonl` — append-only rows, one row per attempt
+- `summary.json` — machine summary
+- `summary.md` — human summary table
+
+Each `attempts.jsonl` row includes:
+- `model_id`
+- `task_id`
+- `run_number`
+- `executor_job_id`
+- `reviewer_job_id`
+- `status`
+- lint/tsc/verdict + token/cost/elapsed fields
+
+## Rerun failed samples only
+
+```bash
+node scripts/run-executor-benchmark.mjs --run-id unitAI-gc2a-r1 --rerun-failed
+```
+
+Runner reads `manifest.json` + `attempts.jsonl`.
+Only samples whose latest status != `success` re-queued.
+Completed success samples remain untouched. Rows append-only for audit trail.
+
+## Lint + tsc recording policy
+
+Executor workflow rule forbids full test suite in harness runs.
+Runner records lint/tsc from reviewer output parsing:
+- `lint_pass|lint`
+- `tsc_pass|tsc --noEmit`
+
+This aligns with benchmark protocol confounder control (`supervisor.test.ts` hang risk).

package/docs/bootstrap.md

@@ -0,0 +1,161 @@
+---
+title: Specialists Bootstrap
+scope: bootstrap
+category: guide
+version: 1.6.1
+updated: 2026-06-24
+synced_at: bf6baf7a
+description: Project bootstrap and installation flow for Specialists.
+source_of_truth_for:
+  - "src/cli/init.ts"
+  - ".mcp.json"
+  - "AGENTS.md"
+  - "CLAUDE.md"
+domain:
+  - bootstrap
+  - mcp
+---
+
+<!-- INDEX: auto-generated by validate_doc.py — do not edit manually -->
+| Section | Summary |
+|---|---|
+| [Install](#install) | From the project root: |
+| [Bootstrap a project](#bootstrap-a-project) | From the project root: |
+| [Force-refresh workflow instructions](#force-refresh-workflow-instructions) | If `AGENTS |
+| [Specialists Workflow](#specialists-workflow) | _no summary_ |
+| [Directory structure](#directory-structure) | Specialists live in ` |
+| [Verify bootstrap](#verify-bootstrap) | Run `specialists status`, `doctor`, `list` to verify install. |
+| [Deprecated commands](#deprecated-commands) | These commands are migration shims only: |
+| [See also](#see-also) | - [workflow |
+<!-- END INDEX -->
+
+# Specialists Bootstrap
+
+> **Alias:** `sp` is a shorter alias for `specialists` — `sp run`, `sp list`, `sp feed` etc. work identically.
+
+`specialists init` is the **sole** project bootstrap command. `specialists init --global` manages a separate XDG-aware global override layer in `~/.config/specialists/user.json`.
+
+Specialists is built on the **[pi coding agent](https://github.com/earendil-works/pi-coding-agent)** and is designed to run alongside **[xtrm-tools](https://github.com/Jaggerxtrm/xtrm-tools)**. pi provides the multi-provider execution layer, lifecycle events, and RPC protocol; xtrm-tools provides the surrounding worktree/session workflow and hook environment. Specialists bootstraps the project-local specialist runtime, workflow instructions, and MCP registration on top of that stack.
+
+## Prerequisites
+
+`specialists init` requires `xt` CLI and a local `.xtrm/` directory. See [docs/installation.md](installation.md) for ordered install steps and runtime-prerequisite details.
+
+## Install
+
+```bash
+npm install -g @jaggerxtrm/specialists
+```
+
+## Bootstrap a project
+
+> **Human-only command.** `specialists init` detects agent sessions (missing TTY, `PI_SESSION_ID`, `PI_RPC_SOCKET`, `SPECIALISTS_JOB_ID`, etc.) and exits with an error. Run it yourself from an interactive terminal — do not invoke it from scripts, hooks, or agent sessions.
+
+```bash
+specialists init
+```
+
+What it does (always safe, idempotent):
+
+1. creates `.specialists/default/` (managed mirror root) and `.specialists/user/` (repo custom layer)
+2. creates runtime dirs `.specialists/jobs/`, `.specialists/ready/`, `.specialists/db/` (observability.db is canonical runtime store)
+3. adds runtime/db paths to `.gitignore`:
+   - `.specialists/jobs/`
+   - `.specialists/ready/`
+   - `.specialists/db/*.db` / `*.db-wal` / `*.db-shm`
+4. injects Specialists section into `AGENTS.md`
+5. registers Specialists in `.mcp.json`
+6. installs canonical hooks into `.xtrm/hooks/specialists/`, then symlinks `.claude/hooks/*`
+7. wires hook commands in `.claude/settings.json`
+8. installs canonical skills into `.xtrm/skills/default/`, verifies `.xtrm/skills/active/` symlinks, and ensures `.claude/skills` + `.pi/skills` root symlinks
+9. runs full FTS memory cache sync from `bd memories` (non-fatal if unavailable)
+
+## Global override layer
+
+```bash
+specialists init --global
+specialists edit --global <name>.execution.model <value>
+```
+
+`--global` manages `~/.config/specialists/user.json`, seeds every shipped specialist with override-allowed defaults, and preserves user-filled values on rerun. It does not bootstrap a repo or touch `.specialists/default/`.
+
+## Package-canonical defaults and drift repair
+
+Current installs resolve Category A runtime assets from the installed package by default. New repositories do **not** need a committed `.specialists/default/` mirror.
+
+Use `.specialists/default/` only for intentional pins or compatibility snapshots. `sp init --sync-defaults` still exists, but it is deprecated because it creates repo-local snapshots that drift from package-canonical assets.
+
+Preferred drift workflow:
+
+```bash
+sp doctor --check-drift
+sp prune-stale-defaults --dry-run
+sp prune-stale-defaults
+```
+
+Use `--keep-diverged` only when you intentionally want to preserve a diverged default snapshot. Put normal customizations in `.specialists/user/` instead.
+
+> **Human-only.** `specialists init` remains an interactive bootstrap command. Agents should not invoke plain `sp init` or compatibility flags from scripts/hooks.
+
+## Directory structure
+
+Specialists live in `.specialists/` in the project root. Skills and hooks are project-local for Claude Code and pi.
+
+```
+.claude/
+├── hooks/         # symlinks -> .xtrm/hooks/specialists/*.mjs
+├── settings.json  # hook wiring (settings.hooks)
+└── skills -> ../.xtrm/skills/active
+
+.pi/
+└── skills -> ../.xtrm/skills/active
+
+.xtrm/
+├── hooks/specialists/   # canonical specialist hooks
+└── skills/
+    ├── default/         # canonical specialist skills mirror
+    └── active/          # active symlinks to default/*
+
+.specialists/
+├── default/             # managed mirror (specialists, mandatory-rules, nodes)
+├── user/                # repo-owned custom specialists
+├── db/                  # runtime db (gitignored)
+├── jobs/                # runtime (gitignored)
+└── ready/               # runtime (gitignored)
+```
+
+Add custom specialists to `.specialists/user/`.
+
+## Managed mirror guidance
+
+- Treat `.specialists/default/` as optional pin/compatibility state, not the normal install source.
+- Do not hand-edit files under `.specialists/default/`.
+- For custom behavior use `.specialists/user/` (specialists) or `config/nodes/` (nodes).
+- Reconcile drift with `sp doctor --check-drift` and `sp prune-stale-defaults`; use `--keep-diverged` only for intentional pins.
+
+## Explicit non-goal
+
+This bootstrap/migration contract does not change backlog-clean behavior or ownership.
+
+## Verify bootstrap
+
+```bash
+specialists status
+specialists doctor
+specialists list
+```
+
+## Deprecated commands
+
+These commands are migration shims only:
+
+- `specialists setup`
+- `specialists install`
+
+They show deprecation warnings and redirect users to `specialists init`.
+
+## See also
+
+- [workflow.md](workflow.md)
+- [mcp-servers.md](mcp-servers.md)
+- [hooks.md](hooks.md)