@jaggerxtrm/specialists 3.17.0 → 3.18.1

package/docs/deploying-alongside.md

@@ -0,0 +1,155 @@
+# Deploying specialists-service alongside an existing compose stack
+
+You have an existing `docker-compose.yml` with services on a custom network. You want to add `specialists-service` as a sidecar that those services can call at `http://specialists-service:8000`. This page is the copyable recipe plus the three undocumented tweaks needed to make it work.
+
+For first-time install on a fresh project, see [`specialists-service-install.md`](specialists-service-install.md). The minimal example at [`docker/compose.example.yml`](../docker/compose.example.yml) is the same shape with placeholders.
+
+For the HTTP contract, see [`specialists-service.md`](specialists-service.md).
+
+## The three tweaks
+
+These are not obvious and are not enforced by the image entrypoint — they are deploy-time decisions.
+
+### 1. `user: "1000:1000"` — host UID/GID for SQLite write perms
+
+The image runs as UID `10001` (`specialists` user) by default. The bind mount at `.specialists/` is owned by your **host** UID — typically `1000:1000` for an interactive Linux user. Without overriding, `sp serve` crash-loops with `EPERM` or `EROFS` the first time it tries to open `observability.db`.
+
+```yaml
+user: "${UID:-1000}:${GID:-1000}"
+```
+
+For non-`1000` hosts, prefer the env-var form so the same compose file works across machines.
+
+### 2. `HOME=/pi-home` — so pi finds its auth + models
+
+The `pi` runtime resolves auth and `models.json` from `$HOME/.pi/`. The image sets `HOME=/home/specialists` for its built-in user, but consumer deployments mount the host's `~/.pi` into `/pi-home/.pi` and need pi to look there instead.
+
+Without this, pi looks at `/root/.pi` (or `/home/specialists/.pi`) and **silently has zero models**. Symptom: every request returns `error_type: specialist_load_error` or `internal` with a model-not-found message in `error`.
+
+```yaml
+environment:
+  HOME: /pi-home
+volumes:
+  - ${HOME}/.pi:/pi-home/.pi:ro
+```
+
+The `:ro` is fine — pi only reads its config.
+
+### 3. RW bind mount of `.specialists/`
+
+The service writes `observability.db` (and SQLite's `-wal` / `-shm` siblings) into `.specialists/db/`. Mount the **directory**, not just the `.db` file, so SQLite can create siblings.
+
+```yaml
+volumes:
+  - ./.specialists:/work/.specialists
+```
+
+The mount is intentionally **read-write**. Read-only breaks observability writes and causes `db_not_writable` from `/readyz`.
+
+## Copyable recipe
+
+This is the production shape from darth-feedor's `ingestion/infra/docker-compose.yml`, slightly de-projectified.
+
+```yaml
+services:
+  specialists-service:
+    container_name: specialists-service
+    image: specialists-service:local      # build separately, or use a published tag
+    user: "${UID:-1000}:${GID:-1000}"
+    stop_grace_period: 30s
+    restart: unless-stopped
+    command: ["serve", "--port", "8000"]
+    env_file:
+      - .env                              # forwards Pi auth tokens / API keys
+    environment:
+      HOME: /pi-home
+    volumes:
+      - ./.specialists:/work/.specialists # rw — observability.db lives here
+      - ${HOME}/.pi:/pi-home/.pi:ro       # ro — pi auth + models.json
+    networks:
+      - merc                              # your existing network
+    # Healthcheck is baked into the image (port 8000 default). Only declare a
+    # compose-level `healthcheck:` block when you override the listen port.
+    deploy:
+      resources:
+        limits:
+          cpus: "1.0"
+          memory: 512M
+
+  # Your other services call it at http://specialists-service:8000
+  my-app:
+    image: my-app:local
+    environment:
+      SPECIALISTS_SERVICE_URL: http://specialists-service:8000
+    depends_on:
+      specialists-service:
+        condition: service_healthy
+    networks:
+      - merc
+
+networks:
+  merc:
+    external: true
+    name: ${MERC_NET_NAME:-mercury_dev_network}
+```
+
+### Build the image
+
+```bash
+git clone https://github.com/Jaggerxtrm/specialists ~/dev/specialists
+cd ~/dev/specialists
+docker build -t specialists-service:local .
+```
+
+The `:local` tag matches the `image:` field above. Once a published tag is available you can swap it out.
+
+### Bring up
+
+```bash
+docker compose up -d specialists-service
+docker compose logs -f specialists-service       # watch for `serve listening on 8000`
+curl -sS http://localhost:8000/readyz             # if you exposed the port; otherwise hit it from a sibling container
+```
+
+For internal-network deploys (no host port), call `/readyz` from a sibling container or use `docker exec`:
+
+```bash
+docker exec my-app curl -sS http://specialists-service:8000/readyz
+```
+
+## Troubleshooting matrix
+
+Symptom → cause → fix.
+
+| Symptom | Likely cause | Fix |
+|---|---|---|
+| Container restart loop, logs show `EACCES`/`EPERM`/`EROFS` opening `observability.db` | Bind mount owned by host UID 1000, container running as UID 10001 | Add `user: "${UID:-1000}:${GID:-1000}"` |
+| `/readyz` returns `503 db_not_writable` | `.specialists/` mounted read-only, or wrong UID | Make the mount rw; check `user:` |
+| Every `/v1/generate` returns `error_type: internal` with `models.json not found` or `no models for ...` | `HOME` not set in environment, pi looks at `/root/.pi` | Add `environment: HOME: /pi-home` AND mount `${HOME}/.pi:/pi-home/.pi:ro` |
+| `/readyz` returns `503 pi_config_unreadable` | `~/.pi` not mounted, or mounted at wrong path | Mount host `~/.pi` to `/pi-home/.pi`; ensure `HOME=/pi-home` |
+| `/readyz` returns `503 empty_user_dir` | `.specialists/user/` has no `*.specialist.json` files | Author or copy at least one script-class spec into `.specialists/user/` |
+| `/readyz` returns `503 invalid_spec_in_user_dir` | One spec file fails schema validation | Run `sp doctor` against the dir, or check container logs for the offending file path |
+| Sibling container can't resolve `specialists-service` hostname | Sibling is not on the same network | Both services must be on the same `networks:` entry |
+| `.specialists/db/observability.db-wal` and `-shm` files appear but the service crashes | SQLite is on a remote filesystem that breaks fcntl locking | Move `.specialists/db/` to a local-disk path |
+| Build fails with `npm` errors during `pi` install | Network restrictions during build | Use `--network=host` on the build, or pre-pull a published `pi` version via `--build-arg PI_VERSION=<tag>` |
+| Healthcheck never passes despite `serve listening` log | Compose-level `healthcheck:` block uses `wget`/`curl` (not in image) and overrides the baked-in node-fetch healthcheck | Remove the compose-level block — image bakes one in. Only override when you change the listen port. |
+
+## Rootless Podman / Fedora / SELinux
+
+For rootless podman or SELinux-enforcing hosts, append `:z` (or `:Z` for unshared bind mounts) to bind mounts:
+
+```yaml
+volumes:
+  - ./.specialists:/work/.specialists:z
+  - ${HOME}/.pi:/pi-home/.pi:z,ro
+```
+
+And pass `--userns=keep-id` to keep host UID matching inside the container so the `user:` override remains correct. See [`specialists-service-install.md`](specialists-service-install.md#rootless-podman--fedora-selinux) for the broader rootless flow.
+
+## Why these tweaks aren't baked into the image
+
+- **`user:`** is intentionally a deploy decision. The image declares its built-in UID via the `org.specialists.uid="10001"` label, but bind-mount permissions belong to the host filesystem and the operator is the only party that knows the right UID for a given deploy.
+- **`HOME=/pi-home`** is consumer-controlled because pi auth resolution needs a stable mount target across hosts. Hard-coding a path inside the image would make the mount target image-version-dependent.
+- **rw `.specialists/`** is fundamental: the service is a stateful sidecar (observability + reload state), and the host operator owns where that state lives.
+
+The recipe above is the working pattern. If you find a fourth deploy-time decision that should be on this page, file a bead with `discovered-from:unitAI-2fz5b`.

package/docs/design/README.md

@@ -0,0 +1,36 @@
+# Design — navigation index
+
+> **2026-06-07 cleanup (ownvq):** chain-templates and substrate canonical moved to `~/dev/xtrm/docs/` (xtrm-product cross-cutting). What stays here is **specialists-package specific** design. Cross-cutting xtrm design lives in the xtrm monorepo.
+
+## 🛠 `roadmap/` — specialists runtime cleanup roadmap
+
+The specialists-runtime cleanup + substrate-alignment effort designed in the 2026-05-27/28 sessions. **Still the actionable work for the specialists package.** Reconciliation with `~/dev/xtrm/docs/temp_roadmap.md` tracked in bead `unitAI-sc03v`.
+
+- **[`roadmap/specialists-roadmap.md`](roadmap/specialists-roadmap.md)** — CANONICAL roadmap. 12 opportunities, 8 phases. Decisions D1–D30 in §0.
+- **[`roadmap/chain-templates/`](roadmap/chain-templates/)** — 13 evidence-backed `bd formula` files (verified vs `bd` binary). Phase 0.a installs them to `~/.beads/formulas/`.
+- **[`roadmap/README.md`](roadmap/README.md)** — roadmap-area navigation.
+- **[`roadmap/history/`](roadmap/history/)** — context-only artifacts that produced the roadmap. Read for context, do NOT act on.
+
+## 📐 Specialists-package design notes (top-level)
+
+- **[`gzrx-tool-catalog.md`](gzrx-tool-catalog.md)** — CANONICAL gzrx manifest + tool catalog design.
+- **[`gzrx-completion-critique.md`](gzrx-completion-critique.md)** — gap analysis for the gzrx completion epic (`unitAI-qujxo`).
+- **[`gzrx-research-notes.md`](gzrx-research-notes.md)** — research findings on agent-runtime tool registries.
+- **[`darth-feedor-migration.md`](darth-feedor-migration.md)** — Darth Feedor migration onto specialists-service.
+
+## 📦 `../archive/` — superseded artifacts
+
+Earlier design iterations and historical planning that have been absorbed into canonical sources (vault, xtrm/docs, or substrate). Includes the issuetracking prototype, plans/, proposals/, other/, iron-review snapshots, friction audits, runtime critiques, substrate review notes, and Tether's predecessor design (`shepherd.md` → now `tether.md` in xtrm/docs/).
+
+## 🏗 Cross-cutting xtrm design (NOT in this repo)
+
+These now live in **`~/dev/xtrm/docs/`** (xtrm monorepo, vault-published):
+
+- `substrate/substrate_design_it.md` — canonical substrate design (rev12, round-3 audit closed via `unitAI-bgyp0`).
+- `substrate/substrate_design_en.md` — EN parallel (rev10 → rev12 propagation tracked in `unitAI-e3w1n`).
+- `substrate/chain_templates.md` + `.html` — canonical chain-template catalog + pipeline.
+- `channels/channels.md` — channel primitive design.
+- `telemetry/*` — agentops + forensic + prometheus contracts.
+- `console/*` — console product contract + handoffs.
+- `tether.md` — always-on context injection (renamed from `shepherd.md`).
+- `monorepo-migration.md` + `state-store-options.md` — migration architecture.

package/docs/design/darth-feedor-migration.md

@@ -0,0 +1,290 @@
+# Darth Feedor — Migration onto specialists-service
+
+**Date:** 2026-04-25
+**Status:** Plan; gated on `specialists-service` v1.0.0 being shipped and locally validated
+**Companion:** `docs/specialists-service.md` (the feature being adopted)
+**Legacy reference:** `docs/script-specialists.md`
+
+This document is the **first reference migration** of a real consumer onto `specialists-service`. The target is the darth feedor VPS stack: ingestion + squawks services that today depend on the legacy YAML-based script-specialists system and `qwen-service`.
+
+It is project-specific. Other consumers can use the structure as a template but should write their own.
+
+---
+
+## Migration shape: clean-room rebuild
+
+- **Not** a big-bang rewrite (legacy and new run side-by-side during the migration).
+- **Not** a backend-only swap (the legacy `SpecialistLoader` and YAML format do not survive).
+- **Yes** a fresh implementation of the consumer side, gradual port one service at a time, single-commit deletion of legacy code at the end.
+
+## Phase gate — feature must exist first
+
+This migration **does not start** until:
+
+1. `specialists-service` is built in this repo (`src/cli/serve.ts`, `Dockerfile.sp-service`, observability writer, script-class Runner branch).
+2. A sample script-class specialist runs end-to-end against the local service.
+3. A reference Python client successfully calls `/v1/generate` and parses the response.
+4. `specialists-service:v1.0.0` is tagged.
+
+If any of these fail, the migration is paused, not worked around.
+
+---
+
+## Phase 0 — Locked decisions for darth feedor
+
+| Question | Answer |
+|---|---|
+| Is `specialists-service` the strategic platform? | **Yes.** |
+| Will services remain orchestrators (multi-stage)? | **Yes.** Orchestration stays in Python for this migration. |
+| Will model transport be owned by services? | **No.** Transport moves into `specialists-service` (via pi). Services lose direct LLM access. |
+| Is `qwen-service` retired, wrapped, or replaced? | **Replaced.** `llm_gateway/` and `~/.qwen` rotation are deleted after Phase 4. |
+| Where do darth feedor specialists live? | `.specialists/user/` (script-class JSON). Legacy `specialists/**/*.specialist.yaml` is converted then deleted. |
+
+These five answers are settled. Do not re-litigate during implementation.
+
+---
+
+## Schema target — what every converted YAML becomes
+
+Every legacy `*.specialist.yaml` becomes a `.specialist.json` file in `.specialists/user/` matching the **script-class** subset of the canonical schema. Authoritative reference: [`docs/authoring.md` § Script-class authoring](authoring.md#script-class-authoring). The service-side runtime constraints come from `compatGuard` in `src/specialist/script-runner.ts` and are visible in [`docs/specialists-service.md`](specialists-service.md).
+
+### Field translation (legacy YAML → new JSON)
+
+The legacy `script-specialists` system used a flat YAML shape; the new schema is nested under `specialist.{metadata,execution,prompt,…}`. The mapping is mechanical:
+
+| Legacy YAML field | New JSON path | Notes |
+|---|---|---|
+| `name` | `specialist.metadata.name` | kebab-case unchanged |
+| `version` | `specialist.metadata.version` | must be quoted string in JSON |
+| `description` | `specialist.metadata.description` | one sentence |
+| (none — implicit) | `specialist.metadata.category` | required; pick from `"analysis"`, `"workflow"`, `"synthesis"`, etc. |
+| `model` | `specialist.execution.model` | provider/model id, e.g. `anthropic/claude-haiku-4-5` |
+| `fallback_model` | `specialist.execution.fallback_model` | optional, recommended cross-provider |
+| `timeout_ms` | `specialist.execution.timeout_ms` | optional, default 120000 |
+| `max_retries` | `specialist.execution.max_retries` | **must be `0` for script class** — caller owns retries |
+| `response_format` (`"json"`) | `specialist.execution.response_format` | `"text"` / `"json"` / `"markdown"` |
+| `prompt` (string) | `specialist.prompt.task_template` | rendered with `$varname` (single-dollar, no braces) |
+| `prompt_normalize` (rolling-context analyst) | second specialist JSON file | schema has one `task_template` per spec — ship a sibling spec for the normalize pass; see Multi-stage section below |
+| `output_schema.required` | `specialist.prompt.output_schema.required` | shape preserved; nested validation is warn-only today |
+| (must add — see Phase 0) | `specialist.execution.interactive: false` | constraint enforced by service |
+| (must add) | `specialist.execution.requires_worktree: false` | constraint enforced by service |
+| (must add) | `specialist.execution.permission_required: "READ_ONLY"` | constraint enforced by service |
+
+If the legacy YAML uses anything not in this table — `tools`, `worktree_required`, `keep_alive`, host shell scripts — that field is out of scope for `specialists-service`. Stay on `sp run` for those specialists, or open a deferred bead (see `unitAI-3k6sa` for skill trust modes).
+
+### Worked example — `mercury-atomic-summarizer`
+
+This is the spec called out by Phase 1. The shape applies to all five YAMLs being ported. The same JSON lives in [`docs/examples/mercury-atomic-summarizer.specialist.json`](examples/mercury-atomic-summarizer.specialist.json) — copy it directly into `.specialists/user/` to start.
+
+```json
+{
+  "specialist": {
+    "metadata": {
+      "name": "mercury-atomic-summarizer",
+      "version": "1.0.0",
+      "description": "Summarizes one news article into one structured row.",
+      "category": "synthesis"
+    },
+    "execution": {
+      "mode": "auto",
+      "model": "anthropic/claude-haiku-4-5",
+      "fallback_model": "openai-codex/gpt-5.4-mini",
+      "timeout_ms": 60000,
+      "interactive": false,
+      "response_format": "json",
+      "output_type": "synthesis",
+      "permission_required": "READ_ONLY",
+      "requires_worktree": false,
+      "max_retries": 0
+    },
+    "prompt": {
+      "task_template": "Summarize the article below into JSON of shape {\"summary\": \"...\", \"controlled_tags\": [...]}.\n\nTitle: $title\n\nContent:\n$content",
+      "output_schema": { "required": ["summary", "controlled_tags"] }
+    }
+  }
+}
+```
+
+The Python adapter (`shared/specialists_client.py` from Phase 1) calls this with:
+
+```python
+client.run("mercury-atomic-summarizer",
+           variables={"title": article.title, "content": article.body})
+```
+
+Variable names must match the `$varname` tokens in the chosen template; missing variables produce `error_type: "template_variable_missing"`.
+
+### Multi-stage specialists (rolling-context analyst)
+
+The schema has one `task_template` per spec; there is no in-spec alternate-template lookup. For Phase 3's analyst stage (initial analysis + normalize pass), ship two specialist files:
+
+```
+.specialists/user/squawk-session-analyst.specialist.json           # task_template = initial analysis
+.specialists/user/squawk-session-analyst-normalize.specialist.json # task_template = normalize the prior synthesis
+```
+
+Python calls each by name:
+
+```python
+client.run("squawk-session-analyst", variables={"context": stage1_input})
+client.run("squawk-session-analyst-normalize", variables={"context": stage1_output})
+```
+
+If the prompts share a system block, factor it into a shared snippet that both specs include verbatim. Two files cost less than a runtime template-lookup feature and they keep the schema strictly 1:1 with the runner.
+
+### Validation before commit
+
+For each converted JSON, run the in-image validator:
+
+```bash
+docker run --rm \
+  -v "$PWD/.specialists/user:/work/.specialists/user:ro" \
+  specialists-service:local \
+  sp validate --target script /work/.specialists/user/mercury-atomic-summarizer.specialist.json
+```
+
+(`sp validate --target script` is not in v1 — see the deferred-items list in `docs/specialists-service-evaluation.md` §12. Until that lands, drop the spec into `.specialists/user/` and start `sp serve` once: the loader surfaces schema errors at request time and `compatGuard` surfaces script-class violations as `specialist_load_error`.)
+
+---
+
+## Phase 1 — Adapter and one converted spec
+
+In the darth feedor (VPS) repo:
+
+1. **`shared/specialists_client.py`** — thin HTTP client wrapping `specialists-service`. Public API is **service-shaped**, not runtime-shaped:
+
+   ```python
+   class SpecialistsClient:
+       def run(self, name: str, variables: dict,
+               template: str = "task_template",
+               timeout_ms: int = 60_000) -> SpecialistResult: ...
+   ```
+
+   `SpecialistResult` matches the existing `qwen_client` return shape (`success`, `output`, `error`, `error_type`, `attempts`) plus `parsed_json` and `meta.trace_id`. Existing call sites change in a minimal way.
+
+   A canonical reference implementation lives at [`clients/python/`](../../clients/python/) — stdlib-only, ~170 LOC, with a smoke test and minimal `pyproject.toml`. Copy `specialists_client.py` into `shared/specialists_client.py` (or `pip install -e clients/python/`); the public API matches this contract exactly.
+
+2. **Convert one YAML to JSON** — start with `mercury-atomic-summarizer`. Drop into `.specialists/user/mercury-atomic-summarizer.specialist.json` per `docs/specialists-service.md` §3.
+
+3. **Smoke test** — point a local script at staging `specialists-service`; verify one real article summarization round-trip works and produces an `articles.summary` row identical (or equivalent) to legacy output on the same input.
+
+4. **Unit tests** for the adapter: HTTP transport, error-type mapping, JSON parsing, missing-required handling, `meta.trace_id` propagation into service logs.
+
+## Phase 2 — Port single-stage consumers
+
+5. **`ingestion/summarizer.py`** — replace `SpecialistLoader + render_prompt + QwenClient` with `SpecialistsClient.run(...)`. Preserve preprocessing, truncation, table injection, controlled-tags handling, and the `articles.summary` write. Run the existing service tests against `specialists-service`.
+
+6. **`ingestion/official_docs.py`** (`official-document-analyzer`) — same pattern. Removes the direct `requests.post()` divergence at the same time. Convert the YAML to JSON in `.specialists/user/`.
+
+After each port, run the corresponding integration tests against a real `specialists-service` instance. No production cutover until the test suite is green on the new path.
+
+## Phase 3 — Port rolling_context (Python orchestration preserved)
+
+7. **`squawks/rolling_context.py`** — port the **invocation layer only**. The three pipeline stages remain three Python `SpecialistsClient.run(...)` calls.
+
+   Stays in Python (these are non-goals for `specialists-service`):
+   - cross-stage state passing (extractor → curator → analyst)
+   - curator fallback to extracted events
+   - synthesis topical grounding heuristics
+   - timestamp resolution and degraded-input metadata
+   - confidence computation
+   - `squawk_rolling_context.processed_data` persistence
+
+8. **Replay harness** — adapt `tests/test_rolling_context_specialists.py` into a regression harness that runs the full pipeline against a real `specialists-service` with recorded fixtures. **This is the safety rail; do not skip.** Rolling context has many implicit operational guarantees that can drift silently.
+
+9. Convert the rolling-context YAMLs to JSON in `.specialists/user/`. The analyst stage becomes two files:
+   - `squawk-rolling-context.specialist.json`
+   - `squawk-event-curator.specialist.json`
+   - `squawk-session-analyst.specialist.json` (initial analysis)
+   - `squawk-session-analyst-normalize.specialist.json` (normalize pass)
+
+   Python calls each spec by name — no `template=` parameter needed.
+
+## Phase 4 — Production cutover
+
+10. Deploy `specialists-service` container to the VPS (sidecar pattern, one per consuming service initially — can collapse to shared later).
+11. Switch `ingestion/summarizer` to the new adapter in production. Monitor `error_rate` / `duration_ms` queries (per `docs/specialists-service.md` §5) for one full operational cycle.
+12. Switch `ingestion/official_docs`. Same monitoring.
+13. Switch `squawks/rolling_context`. Watch the replay harness output; compare a week of synthesized output against legacy output on the same squawks before declaring success.
+
+## Phase 5 — Decommission
+
+After all three consumers run on `specialists-service` cleanly for one week:
+
+14. Delete `shared/specialist_system/` (loader + schema).
+15. Delete `shared/qwen_client.py`.
+16. Delete `llm_gateway/` and remove `qwen-service` from `ingestion/infra/docker-compose.yml`.
+17. Delete `specialists/**/*.specialist.yaml`.
+18. Update or remove `docs/guides/qwen-service-integration-guide.md` and `specialists/SPECIALIST_SYSTEM_IMPLEMENTATION.md`.
+19. Remove `~/.qwen` references from operator skills/runbooks.
+
+After Phase 5 the VPS has no legacy script-specialist surface left.
+
+---
+
+## Deferred — externalizing rolling_context orchestration
+
+Whether rolling_context's three-stage chain should eventually move out of Python and into a runtime feature is a **separate future evaluation**. If revisited, the candidate is the **node-coordinator** system in this repo (`.specialists/default/nodes/`), not `specialists-service`. That feature is built for chained specialist work; `specialists-service` is not.
+
+This deferral is intentional and is the only future expansion point in this plan.
+
+---
+
+## Risks and mitigations
+
+| # | Risk | Mitigation |
+|---|---|---|
+| 1 | Hidden behavior regression in rolling_context (retry-once, curator fallback, synthesis rejection, EOD reset) | Replay harness in Phase 3, week-long parallel comparison in Phase 4 |
+| 2 | Losing service-owned observability during transition | `meta.trace_id` returned to Python; service logs correlate to `observability.db` rows |
+| 3 | Partial migration leaving split-brain | Phase 5 deletes all legacy artifacts in one commit, only after Phase 4 is green |
+| 4 | Over-generalization of `specialists-service` to absorb orchestration | §7 non-goals in `docs/specialists-service.md` are enforced; orchestration relocation is explicitly deferred |
+| 5 | Validation gaps persisting under new branding | Shallow-required validation is documented; strict-schema validation is a follow-up feature, not in this scope |
+| 6 | Pi backend instability replacing qwen-service instability | `specialists-service` is the only place that talks to pi; if pi changes, only one container changes |
+
+---
+
+## File impact
+
+### darth feedor repo
+
+**New:**
+- `shared/specialists_client.py`
+- `tests/test_specialists_client.py`
+- `.specialists/user/mercury-atomic-summarizer.specialist.json`
+- `.specialists/user/official-document-analyzer.specialist.json`
+- `.specialists/user/squawk-rolling-context.specialist.json`
+- `.specialists/user/squawk-event-curator.specialist.json`
+- `.specialists/user/squawk-session-analyst.specialist.json`
+- rolling-context replay fixtures
+
+**Modified:**
+- `ingestion/summarizer.py`
+- `ingestion/official_docs.py`
+- `squawks/rolling_context.py`
+- `ingestion/infra/docker-compose.yml` (add `specialists-service`, remove `qwen-service`)
+- legacy docs (or delete and replace with pointer)
+
+**Deleted (Phase 5):**
+- `shared/specialist_system/`
+- `shared/qwen_client.py`
+- `llm_gateway/`
+- `specialists/**/*.specialist.yaml`
+
+### This repo
+
+No changes for the migration itself. All `specialists-service` work happens in Phase 1 of the feature build (see `docs/specialists-service.md`).
+
+---
+
+## Validation checklist before declaring the migration done
+
+- [ ] `mercury-atomic-summarizer` running on `specialists-service` in production for ≥ 7 days with no regression in `articles.summary` quality vs. legacy baseline
+- [ ] `official-document-analyzer` likewise
+- [ ] Rolling context replay harness passes on recorded fixtures
+- [ ] Rolling context running in production for ≥ 7 days with parallel comparison vs. legacy showing no behavioral drift
+- [ ] All five legacy YAMLs deleted
+- [ ] `qwen-service` and `llm_gateway/` removed from docker-compose
+- [ ] No `~/.qwen` references in active operator runbooks
+- [ ] `observability.db` queries (per `docs/specialists-service.md` §5) show healthy error rates
+
+When every box is ticked, close the migration.

package/docs/design/roadmap/README.md

@@ -0,0 +1,32 @@
+# Roadmap — specialists-runtime cleanup + substrate-alignment
+
+Workaround/adaptation we designed to bridge today's runtime to substrate without waiting for substrate to land. **This is the actionable work for the next ~3–4 days of specialists-auto execution.**
+
+## What to read when
+
+| When | File | What it gives you |
+|---|---|---|
+| **You're picking up the work** | bd bead `unitAI-wxi9e` (P0) | Handoff: pointer index + 3 meta-phases (validation → planning → auto-execution) + per-phase smoke checkpoints + do-not list |
+| **You want the plan** | [`specialists-roadmap.md`](specialists-roadmap.md) | Canonical roadmap — 12 opportunities, 8 phases (0–7), decisions D1–D30, parallelization map, friction catalog, reads-forward to substrate |
+| **You're about to install chain templates** | [`chain-templates/README.md`](chain-templates/README.md) + the 13 `.formula.json` | Phase 0.a target — copy to `~/.beads/formulas/`; substrate-aligned shape; post-pour wire-edges spec |
+| **You're cross-checking design history** | [`history/substrate-reconciliation.md`](history/substrate-reconciliation.md) | The substrate-author's decision-delta against the roadmap. Already applied to the canonical roadmap. Read for context only. |
+| **You're cross-checking handoff context** | [`history/handoff-from-substrate-design.md`](history/handoff-from-substrate-design.md) | The handoff from the substrate-design author to the specialists-runtime work. Names the open questions deferred to the next-agent-with-code-visibility. Already addressed in the roadmap. |
+
+## What this is NOT
+
+- Not substrate design — that's [`../substrate/`](../substrate/). The roadmap reads forward to substrate sections.
+- Not implementation — the roadmap is the plan; implementation happens via the handoff bead.
+- Not retrofit of existing artifacts (specialists, beads, etc.) — the roadmap is forward-only where retrofit cost would dominate.
+
+## Key inputs (live system state, not files in this directory)
+
+- `config/specialists/*.specialist.json` — 19 package-tier specialists (the dispatch catalog)
+- `config/mandatory-rules/` — 20 rule files (template_sets injection)
+- `config/skills/` — operator-facing skills (canonical sources)
+- `src/specialist/` + `src/cli/` — runtime + CLI surface to modify
+- `~/.beads/formulas/` — chain template install target (Phase 0.a populates from `chain-templates/`)
+- `bd memories <keyword>` / `bd recall <key>` — persistent project memory (576+ entries)
+
+## Active bead
+
+`unitAI-wxi9e` (P0) — HANDOFF: specialists-runtime cleanup + substrate-alignment — fresh-session validation → planning → specialists-auto execution.