@ngocsangairvds/vsaf 4.1.9 → 4.1.10

package/skills/vds-skill/runtime/docs/p8-preflight-evidence/pgbouncer-search-path-empirical.md

@@ -0,0 +1,110 @@
+# PgBouncer `search_path` Empirical Probe (post-deploy)
+
+**Date**: 2026-04-30
+**Probe context**: live `vds-platform` deployment, master @ `a9d342b5`, PgBouncer 1.25.1, postgres 18 (pgvector/pgvector:0.8.2-pg18)
+**Spec reference**: ecosystem-infrastructure-evolution FR-15, ADR-INF-13
+
+## Why this document exists
+
+Spec FR-15 prescribes per-stanza `connect_query='SET search_path TO <schema>, public'` to pin the search_path "at backend startup time." The FR-9 `verify-infra-topology` gate verifies *declarations* (stanzas exist, pool_mode=session, etc.) but explicitly defers runtime correctness: "this verifies declarations only; per-stanza connect_query search_path correctness needs a per-stanza psql probe." This document is that per-stanza probe.
+
+## Method
+
+Connections were issued through each of the four PgBouncer stanzas (`vds_audit`, `vds_memory`, `vds_progress`, `vds_sched`) with the auth_query-resolved credentials in `~/.vds/.env`. For each stanza:
+
+1. Force a fresh backend via `RECONNECT <stanza>` issued to the pgbouncer admin DB.
+2. Open a new client session through the stanza.
+3. Capture `current_database()`, `current_user`, `current_schema()`, `current_setting('search_path')`.
+4. Issue a no-op `SELECT 1` and release the session.
+5. Open a *second* client session against the same backend (recycled from the pool).
+6. Capture the same four values.
+
+For the `vds_memory` stanza only, additionally probe `pgvector` extension visibility: `SELECT extname FROM pg_extension WHERE extname='vector';` plus `SELECT '[1,2,3]'::vector;`.
+
+## Per-stanza results
+
+Each stanza was probed in two states: **fresh** (first client on a brand-new backend post-RECONNECT) and **recycled** (second client on the same backend after the first client released it).
+
+### `vds_audit`
+| state | current_database | current_user | current_schema | search_path |
+|---|---|---|---|---|
+| fresh | `vds_platform` | `postgres` | `audit_state` | `audit_state, public` |
+| recycled | `vds_platform` | `postgres` | `public` | `"$user", public` |
+
+### `vds_memory`
+| state | current_database | current_user | current_schema | search_path |
+|---|---|---|---|---|
+| fresh | `vds_platform` | `postgres` | `memory` | `memory, public` |
+| recycled | `vds_platform` | `postgres` | `public` | `"$user", public` |
+
+pgvector extension probe (recycled state):
+- `extname` row returned: `vector` (extension is installed in `public`).
+- `'[1,2,3]'::vector` parses successfully.
+
+The pgvector type resolves on recycled backends *only because* `public` remains in the cluster default `search_path`. If `public` were removed from defaults, `vector_l2_ops` would become invisible on recycled backends.
+
+### `vds_progress`
+| state | current_database | current_user | current_schema | search_path |
+|---|---|---|---|---|
+| fresh | `vds_platform` | `postgres` | `progress` | `progress, public` |
+| recycled | `vds_platform` | `postgres` | `public` | `"$user", public` |
+
+### `vds_sched`
+| state | current_database | current_user | current_schema | search_path |
+|---|---|---|---|---|
+| fresh | `vds_platform` | `postgres` | `sched` | `sched, public` |
+| recycled | `vds_platform` | `postgres` | `public` | `"$user", public` |
+
+## Pool-mode confirmation (`SHOW POOLS`, `SHOW STATS_TOTALS`)
+
+All four app stanzas confirmed `pool_mode=session`. Live activity at probe time:
+
+| stanza | cl_active | sv_active | sv_idle |
+|---|---|---|---|
+| `vds_audit` | 0 | 0 | 1 |
+| `vds_memory` | 9 | 9 | 0 |
+| `vds_progress` | 0 | 0 | 1 |
+| `vds_sched` | 6 | 6 | 0 |
+
+`SHOW STATS_TOTALS` shows traffic on all four stanzas. `vds_sched` is busiest (9588 transactions); `vds_memory` second (239 transactions, 1192 queries).
+
+## Root cause of the recycled-state regression
+
+PgBouncer's `connect_query` runs **once per backend at backend startup**. When a client session ends, PgBouncer issues `server_reset_query` (default: `DISCARD ALL`) before returning the backend to the pool. `DISCARD ALL` resets `search_path` to the cluster default (`"$user", public`). When PgBouncer assigns that recycled backend to a new client, **it does not re-run `connect_query`** — it assumes the backend is in a clean baseline state.
+
+The net effect: only the *first* client of each backend sees the stanza-specific `search_path`. Every subsequent client on a recycled backend inherits the cluster default until the next `RECONNECT` or backend rotation.
+
+## Verdict
+
+| Property | Status |
+|---|---|
+| Per-stanza `connect_query` syntactically applied | ✅ Yes (verified on fresh backends) |
+| Per-stanza `search_path` pinning persists across session releases | ❌ No (wiped by `DISCARD ALL`) |
+| pgvector visibility from `vds_memory` | ✅ Yes (incidentally — `public` in cluster default search_path) |
+| `pool_mode=session` for all stanzas | ✅ Yes |
+
+## Production blast radius
+
+VDS application code schema-qualifies all table references (e.g. `audit_state.checkpoints`, not just `checkpoints`) per Phase 220 review. Production correctness is therefore independent of `search_path` pinning. The `connect_query` mechanism is **best-effort defense-in-depth**, not a hard guarantee.
+
+Code paths that rely on unqualified-name resolution would be at risk on the second-or-later assignment of any backend. A grep pattern to find such risk surface:
+
+```
+rg "(?<![\w.])(checkpoints|memories|memory_fragments|knowledge_documents|workflow_state|workflows|reports|cron_schedules)(?![\w.])" \
+  audit_orchestrator/ memory_orchestrator/ progress_report_orchestrator/ scheduler_orchestrator/
+```
+
+False positives are expected (string literals, comments); manual triage required.
+
+## Recommendations
+
+1. **Update spec FR-15 and ADR-INF-13** to describe `connect_query` as best-effort defense-in-depth, not a persistent pin. The actual contract is: "fresh backends start with the stanza search_path; recycled backends inherit cluster defaults; production correctness comes from schema-qualification."
+2. **Do NOT remove `DISCARD ALL` from `server_reset_query`** — it is the standard reset-to-baseline mechanism for session-pooled backends and protects against far worse leakage (prepared statements, advisory locks, SET LOCAL residue). The bug is upstream PgBouncer's choice to skip `connect_query` on recycle, not our reset query.
+3. **PgBouncer `track_extra_parameters` does not apply** because clients do not issue `SET search_path` — the pinning was supposed to happen at the pooler. Tracking client-issued SETs cannot recover values the client never set.
+4. **Future R12 (per-stanza tx-mode promotion)** is unaffected by this finding — transaction mode releases backends per-transaction, so the `connect_query`-vs-recycle gap manifests at a different cadence but with the same root cause and same mitigation (schema qualification).
+
+## Artifacts
+
+- Probe commands and raw output: collected during W1-B research wave 2026-04-30 by Lane B teammate.
+- pgbouncer.ini (probed config): `docker/infrastructure/pgbouncer/pgbouncer.ini` at master @ `a9d342b5`.
+- init-schemas.sql (role-schema mapping): `docker/infrastructure/init-schemas.sql`.

package/skills/vds-skill/runtime/docs/p8-preflight-evidence/pgvector-set-local-audit.md

@@ -0,0 +1,51 @@
+# pgvector SET hnsw.ef_search Pre-flight Audit (TSK-8.0.3)
+
+**Date**: 2026-04-27
+**Purpose**: Verify zero plain `SET hnsw.ef_search` (without `LOCAL`) sites exist in `memory_orchestrator/`. The rule is forward-looking: under v2.15.3 session-mode pooling, plain `SET` is safe (1:1 client:backend), but per-stanza R12 promotion to transaction mode would expose plain `SET` to cross-client GUC leakage. Auditing now keeps the codebase ready for that future-decision without code changes.
+
+## Verdict
+
+✅ **CLEAN — zero current violations**
+
+## Evidence
+
+### Plain `SET hnsw.ef_search`
+
+```bash
+$ rg -n 'SET\s+hnsw\.' memory_orchestrator/
+(0 matches)
+```
+
+### Any reference to `hnsw.ef_search` or bare `ef_search`
+
+```bash
+$ rg -n 'hnsw\.ef_search|ef_search' memory_orchestrator/
+(0 matches)
+```
+
+`memory_orchestrator/` does not currently tune `hnsw.ef_search` at all — neither plain nor `SET LOCAL`. HNSW indexes exist (verified: `CREATE INDEX … USING hnsw` in `memory_orchestrator/alembic/versions/0003_create_memory_vectors_if_missing.py:38-40` and `memory_orchestrator/database/postgres_manager.py:309-310, 357-358`), but search code uses Postgres' default `ef_search=40`.
+
+## Forward enforcement
+
+Even though current code is clean, the rule still applies for future code. Recommended enforcement options:
+
+1. **Lint rule** (preferred — fail-fast at CI time): `ast-grep` rule that flags `SET hnsw.ef_search` *not* preceded by `LOCAL` keyword in `memory_orchestrator/`. Needs careful pattern: only flag SQL strings, not Python identifier `ef_search`.
+
+2. **Integration test (post-R12)**: when any stanza is promoted to `pool_mode=transaction` via per-database override, add a test that sets `hnsw.ef_search` via plain `SET` from one client and asserts the next client on the same backend doesn't observe the leaked value. Catches both authored mistakes and library regressions.
+
+3. **Code review checklist** in `infrastructure-evolution-skill`: when adding any `SET <param>` SQL, verify it's `SET LOCAL` inside a transaction, OR encoded in DSN options / pgbouncer.ini `connect_query`.
+
+For P8 scope, no `connect_query`-embedded default is needed — pgvector's built-in `ef_search=40` is the working default; the rule above keeps future code session-mode-and-transaction-mode-portable.
+
+## Note: pgvector default ef_search performance
+
+Postgres pgvector ships with `ef_search=40` (per pgvector README). This is fine for typical workloads. If memory_orchestrator's vector search ever needs tuning past this, the correct pattern (post-P8) is to issue `SET LOCAL hnsw.ef_search = N` inside the query transaction, never plain `SET`. Confirmed pgvector docs: https://github.com/pgvector/pgvector#querying
+
+## Conclusion
+
+WP-8.0.3 audit closes with zero violations. No code migration needed. Forward-enforcement rule documented (lint or test) for future code; will be added in WP-8.6 (verify-infra-topology +6 checks) or as a separate static-analysis rule.
+
+## References
+
+- Spec: `.gpt-5.4/specs/ecosystem-infrastructure-evolution/design.md` §ADR-INF-13 caveat 1
+- pgvector README: https://github.com/pgvector/pgvector

package/skills/vds-skill/runtime/docs/p8-preflight-evidence/topology-decision-session-mode.md

@@ -0,0 +1,57 @@
+# P8 Topology Decision — Session-Mode Uniform (TSK-8.0.4 / Wave 1 Addendum)
+
+**Date**: 2026-04-27
+**Purpose**: Document the topology decision after Wave 1 pre-flight surfaced a NEW HIGH BLOCKER (audit_orchestrator runtime session-scoped advisory locks) and the team's preference for "consistent setup across ecosystem".
+
+## Resolution
+
+**Chosen topology**: PgBouncer session-mode global default + per-database stanzas with `connect_query`.
+
+This supersedes the Options A/B/C analysis at the bottom of `alembic-and-runtime-advisory-locks.md`. None of those options were chosen — instead, a 4th option emerged from follow-up research: **session-mode-uniform** is the simplest topology that satisfies every hard constraint with zero exceptions and zero service-code changes.
+
+## Why session mode
+
+Session mode is the canonical answer to all 4 of VDS's hard constraints:
+
+| Constraint | Session-mode behavior |
+|---|---|
+| DBOS LISTEN/NOTIFY (scheduler event bus) | ✅ Works — session held per-client; LISTEN survives `COMMIT` |
+| audit_orchestrator session-scoped advisory locks | ✅ Works — `pg_try_advisory_lock` + multi-transaction span supported |
+| asyncpg `statement_cache_size=512` | ✅ Works — full cache benefit; same as direct DSN |
+| alembic `pg_try_advisory_xact_lock` | ✅ Works — tx-scoped is a strict subset of what session mode permits |
+
+Plus all the secondary tx-mode foot-guns evaporate: WITH HOLD cursors, server-side cursors, temp tables, `application_name` visibility, query cancellation race, blanket `SET` rules — all just work.
+
+## What we forfeit by NOT using transaction mode
+
+- **Multiplexing**: session mode is 1:1 client:backend. Transaction mode would let ~20 clients share a single backend across transactions — a 5-20× client fan-out per backend.
+- **Lower max_connections budget**: tx-mode could justify dropping `max_connections` from 500 to ~150-200 (multiplexing absorbs the difference). Session mode keeps it at 500.
+
+These costs are acceptable at VDS scale (5 services × in-app pool ~30 = ~150 sustained backends; well within 500 with ~70% headroom).
+
+## Future R12 escalation path
+
+When measurement shows benefit, *individual stanzas* can be promoted to `pool_mode=transaction` via per-database override (one config-line change per stanza). Trigger condition: `SHOW POOLS` shows sustained `cl_waiting>0` on memory or progress stanzas after 90 days of production observation. Audit + sched stanzas remain session-mode permanently per their session-scoped state requirements.
+
+This converts the "should we use transaction mode?" question from an architecture decision (made in spec, hard to reverse) into an operational one (made post-deploy with measurement, reversible per-stanza).
+
+## What changed in v2.15.3 vs v2.15.2
+
+| Change | Impact |
+|---|---|
+| Pool mode `transaction` → `session` | Eliminates 4 tx-mode hard constraints; 6 secondary caveats irrelevant |
+| Dropped `_DIRECT_DSN` parallel env vars | Single `VDS_<SVC>_DSN` per service; no parallel scaffolding |
+| `max_connections` retune 500→200 deleted | Stays at 500; 70% headroom in v2.15.3 |
+| `resolve_database_url(direct: bool = False)` flag | Not added; existing helper API works unchanged |
+| DBOS dual-pool config | Not added; existing `system_database_url`-only config works unchanged |
+| WP-8.3 (alembic-migrate → DIRECT_DSN) | Deleted; alembic routes through pooler same as services |
+| WP-8.5 (max_connections retune) | Deleted |
+| Multi-skill refactor | 5 mandatory skill bumps → 4 (progress-report no longer needs env-var change) |
+| Rollback ordering | Simplified — no Postgres restart required |
+
+## References
+
+- Spec: `.gpt-5.4/specs/ecosystem-infrastructure-evolution/design.md` §ADR-INF-13 v2.15.3
+- Predecessor evidence: `dbos-listen-notify.md`, `alembic-and-runtime-advisory-locks.md`, `pgvector-set-local-audit.md`
+- Research transcript IDs: a0182d2bbf9cec6df (initial), af7c71a11326c0d03 (deep), a0e3d69364a42cdf6 (implementation-ready), ac8dd55aea05c00e1 (consistent-topology re-research)
+- Authoritative sources: pgbouncer.org/config.html, pgbouncer.org/features.html, pgbouncer.org/faq.html, jpcamara.com/2023/04/12/pgbouncer-is-useful.html

package/skills/vds-skill/runtime/docs/phases/CHANGELOG.md

@@ -0,0 +1,103 @@
+# Phase Changelog Archive
+
+Extracted from AGENTS.md lines 1-66.
+
+## Phase 225 — Pluggable Embedding + Per-Agent Provider Config (DEPLOYED 2026-05-02)
+
+Merged to vds-scripts master at `41798074` (W1-W7 fast-forward chain on top of `b9d7a08f`); companion vds-skills `vds-ai-memory-skill` v3.10.0 at `027fb85`. Spec `.gpt-5.4/specs/vds-ai-memory/` v20.6.1.
+
+**Scope** (FR-225.1..23 + FR-226.1..15 + NFR-225.1..15 + NFR-226.1..4 — 50 spec items):
+
+- **Track-A — Pluggable embedding SDK** hoisted from `memory_orchestrator/.../embedding_service.py` into `vds_agent_core.llm.embedding`: `EmbeddingClient` Protocol + 9-provider registry (`ollama`, `openai`, `cohere`, `voyage`, `mistral`, `jina`, `litellm`, `local`, `hash`) + `EmbeddingSettings(BaseSettings)` factory + auto-routing from `<provider>/<model>` strings. Default provider Ollama (host-native, Metal GPU on macOS); cloud failover via OpenAI matryoshka built-in.
+- **Track-B — Per-agent provider config** via 3-layer cascade (defaults → orchestrator subclass → agent runtime). `LLMSettings.for_agent(agent_id)` + `EmbeddingSettings.for_agent(agent_id)` factories. Per-agent overrides via env (`VDS_AGENT_<ID>_EMBED__*`), API (`AgentRegistrationRequest`), or programmatic (`overrides=...`). Credential-reference indirection (`${env:VAR_NAME}`) with NFR-225.12 allowlist.
+- **W4 deploy hygiene**: `memory_orchestrator/Dockerfile` drops the bge-m3 sentence-transformers bake; `pyproject.toml [embeddings]` drops `sentence-transformers`; `docker-compose.services.yml` adds `VDS_AGENT_CORE_EMBED__*` defaults + `extra_hosts: ['host.docker.internal:host-gateway']` for Linux portability.
+- **W7 dedup consolidation**: `vds_agent_core.security.credentials` (NEW) consolidates two divergent regex implementations into a single canonical sanitizer covering Bearer / `sk-` / `hf_` prefixes plus URL query params (`?api_key=...`) plus JSON body fields (`"api_key":"..."`).
+
+**ADRs landed**: ADR-98 (embedding hoisted to vds_agent_core; default Ollama), ADR-99 (registry-based provider routing + LiteLLM-style auto-routing), ADR-100 (3-layer cascading settings).
+
+**Operational impact**:
+
+- Image size: `vds-memory:1.0.0` **27.6 GB → 10.4 GB** (62% reduction; ML stack moved to host Ollama).
+- Build time: ≤ 2m 39s on warm BuildKit cache (target was ≤ 5 min).
+- Container Ollama round-trip: 1024-dim verified live via `host.docker.internal:11434`.
+- `vds-cli memory` subcommand surface intact (19 commands; 0 regressions).
+- 277 new Phase 225 tests + 1163 vds_agent_core total tests; ~99% Phase 225 surface coverage.
+
+**Operator action required for cloud failover**:
+
+- For OpenAI: set `VDS_AGENT_CORE_EMBED__PROVIDER=openai`, `MODEL_STANDARD=text-embedding-3-large`, `DIMENSION=1024` (matryoshka), root `OPENAI_API_KEY`.
+- For dev hosts without Ollama: set `VDS_AGENT_CORE_EMBED__FAIL_OPEN=true` (fall back to `HashEmbeddingClient`).
+- For per-agent provider mixing: see `vds_agent_core/docs/embedding.md` §Per-Agent Provider Config.
+
+**W8 post-merge fix**: `service/api/v1/embedding.py` `re-embed` + `embedding_status` endpoints fixed to read provider/model/dimension via `embedding_service._settings.*` instead of the W4-deleted `_provider`/`_model`/`_dimension` attributes (would have AttributeError'd in production on the first non-dry-run re-embed call).
+
+**Out of scope** (deferred): Phase 227 per-agent operator tooling stub (FR-227.1..6, trigger-gated); Phase 221 (VAMI-031 WebSocket subprotocol), Phase 222 (VAMI-032 mem0 v3 cloud entity-scope migration) — both PROPOSED stubs awaiting trigger conditions per spec.
+
+---
+
+# AGENTS.md — VDS Scripts Workspace
+
+> Live validation 2026-04-28: **2,610+ tests, 0 failures** across 5 packages. Scheduler at `a1b2c3d4e5f8` (Phase 2 columns live). 10/10 Docker healthy.
+
+Operational rules for `WHO-project/vds-scripts`.
+
+Merged mainline runtime baseline is Phase 171 implemented (schema contract hardening, startup gating, gradual dispatch ramp-up, and dispatch observability all merged into the current mainline runtime).
+Phase 168 fixes dispatch state finalization reliability:
+- **Atomic `finalize_dispatch_run()`**: single-transaction store method replaces N+1 separate DB ops — run, lanes, and events written atomically with full rollback on failure.
+- **`_finalization_store` preservation**: saved before the dispatch loop; survives mid-run `_dispatch_store` nulling at degradation time so finalization always writes.
+- **Resilient wrapper**: `_finalize_dispatch_run()` retries up to 3 times (2s, 4s backoff); final failure logs at `error` level — no silent swallow.
+- **Schema fix**: `_claim_resumable_dispatch_lanes()` now sets `status = 'running', claimed_at = NOW()` (not `'claimed'`); stale-claim recovery added to CTE WHERE.
+- **`doctor repair-dispatch`**: new subcommand force-closes stuck runs via `finalize_dispatch_run()` with `termination_reason='force_closed_by_doctor'`; uses `started_at` staleness (no `updated_at` column).
+- **Parent finalization**: `workflow-projects` parent path replaced bare `except Exception: warning` with `_finalize_dispatch_run()` retry wrapper.
+Phase 171 adds gradual dispatch ramp-up for safer startup behavior:
+- **Gradual capacity scaling**: `_fill_capacity()` starts with `ramp_up_slots` capacity instead of `max_concurrent`, increasing by `ramp_up_slots` every `ramp_up_interval` seconds until `max_concurrent` is reached.
+- **CLI params**: `--ramp-up-slots` (min=0, default=0) and `--ramp-up-interval` (min=1, default=30). `ramp_up_slots=0` preserves Phase 170 behavior.
+- **Ramp-up tick placement**: fires on both job completion and poll timeout — capacity grows on schedule even when all active slots are running.
+- **Safer default stagger**: `submission_stagger_ms` default raised from 100ms to 200ms for safer burst behavior.
+- **Logging**: `dispatch_ramp_up_capacity_increased` structured log event on each capacity bump with `new_capacity` and `max_concurrent`.
+- **Tests**: 23 new tests in `test_dispatch_ramp_up_phase171.py`.
+Phase 170 hardens the audit-state schema contract for DB-native dispatch:
+- **Schema contract**: `_DISPATCH_SCHEMA_CONTRACT` defines 7 dispatch tables + `dispatch_lanes.claimed_at` as blocking contract items; evaluated during every `PostgresStateStore.__init__`.
+- **Safe additive repair**: missing `dispatch_lanes.claimed_at` is repaired with `ALTER TABLE … ADD COLUMN IF NOT EXISTS … TIMESTAMPTZ` before dispatch can proceed; idempotent and safe to run repeatedly.
+- **Startup gating**: `dispatch_schema_gate(store)` raises `DispatchSchemaIncompatibleError` for `incompatible_blocking` or `unreachable` — hard fail before any dispatch state is created.
+- **Resume/parent-dispatch gate**: same gate applied to the resume claim path (`_claim_resumable_dispatch_lanes`) and `workflow-projects` parent dispatch startup independently.
+- **`doctor schema` command**: `vds-cli audit doctor schema [--json]` reads `_schema_compat_result` with no extra DB round-trip; JSON includes `status`, `dispatch_safe`, `db_reachable`, `missing_tables`, `violations`, `repaired_items`, and `remediation`; exits 0/1/2.
+- **`doctor repair-dispatch` pre-check**: schema compatibility validated before repair begins; `incompatible_blocking` or `unreachable` exits 2 with `schema_compatibility.blocking_reasons` in JSON.
+Phase 169 adds dispatch observability:
+- **Dispatch event contract**: DDL widened to 9 event types (adding `lane_closed_by_doctor`, `dispatch_doctor_repair`); `_DISPATCH_SCHEMA_CONTRACT` entry for `event_type` column.
+- **Heartbeat**: `heartbeat_at` column on `dispatch_runs`, written every 10 completions; `dispatch_run_status` VIEW includes heartbeat; `update_dispatch_run_heartbeat()` store method.
+- **Utilization persistence**: `run_summary["utilization"]` populated from `_DispatchUtilizationTracker.summary()` and persisted to `dispatch_runs.metadata_json`.
+- **Progress log**: `dispatch_progress_summary` fires at 50-completion or 300-second boundary with `completed_total`, `pending_remaining`, `active_slots`, `slot_utilization_pct`.
+- **Dead-run detection**: `vds-cli audit doctor dispatch-status --dead-threshold-seconds N` flags `potentially_dead` runs; heartbeat age formatting in table output.
+- **Tests**: 20+ new tests across 5 test files.
+Phase 167 adds dynamic late-binding profile assignment (`rows_per_shard==1` only), profile-agnostic `dispatch_job_id`, pre-dispatch row exclusion via shared `_build_instruction_analysis_text` helper, time-weighted utilization telemetry (`_DispatchUtilizationTracker`), and full regression verification.
+Phase 166 adds route-aware provider failure classification (`ROUTE_PROVIDER_UNAVAILABLE`), failover-only strategy for provider-route credential gaps (skip same-provider retry, go directly to cross-profile failover), truthful synthesis fallback telemetry via `SynthesisFallbackCause` enum, prompt-level retry/failover counters in `resilience_summary`, and structured warnings for unclassified 400-class provider errors.
+Phase 165 adds consolidated tmux runner tooling, dashboarding, script archival, and explicit manual continuation guidance on top of the merged Phase 164 runtime.
+Phase 164 adds structured outcome interpretation and exclusion tracking to the dispatch and summary surfaces:
+- **DispatchOutcomeClassification enum**: structured outcome interpretation replacing ad-hoc string matching — classifies every dispatch result into a canonical category for downstream consumers.
+- **Exclusion tracking**: `workflow-summary.json` now records explicit exclusion metadata and aggregated exclusion summaries for scope, readiness, analysis-excluded, profile-unavailable, and checklist-parseability paths so zero-work and degraded outcomes stay operator-visible.
+- **Timeout diagnostics**: dispatch summary includes `timeout_kind` (hard/soft/provider) and `partial_rows_count` for rows that started evaluation but did not complete.
+- **Zero-work reason classification**: when a workflow-project run completes with no evaluated rows, the summary includes a structured `zero_work_reason` field explaining why (no repos in scope, all rows excluded, provider unavailability, etc.).
+Post-Phase 164 fixes (merged to mainline):
+- **Sharded merge checklist propagation**: `merged_rows_payload` now spreads `**row.row_data` as the base dict in both `_load_and_merge_runs` and `_merge_row_sharded_repo_runs`, ensuring full checklist fields (requirement_text, score_breakdown, evidence_refs, provenance) flow through to `_materialize_project_repo_report_dirs`. This enables `audit-checklist.json` materialization and full 5-section Confluence reports (checklist_core + checklist_details) for `--from-db-runs` and row-sharded merge paths.
+- **Incremental upload sidecars**: `_materialize_project_repo_report_dirs` now writes `audit-checklist.delta.json` and `audit-checklist.meta.json` alongside `audit-checklist.json`, enabling `--incremental-upload` section-level patching instead of full page rewrites for materialized reports.
+- **Row patching for model compatibility**: Materialization patches rows with default `provenance` and `score_breakdown.raw_score` when legacy DB rows lack these fields, preventing `AuditChecklist` validation failures.
+Phase 163 is consolidated into the mainline baseline.
+Phase 162 fixes the Confluence probe consistency mismatch between doctor diagnostics and workflow startup auto-offline behavior, and adds:
+- **Shared Confluence connectivity contract**: `confluence_connectivity.py` extracts endpoint probing into a reusable helper shared by doctor and workflow startup.
+- **Endpoint-set `any_reachable` policy**: startup uses the same multi-endpoint probe as doctor; false offline activation is eliminated when any configured endpoint is reachable.
+- **Improved startup diagnostics**: structured logs include `decision_basis`, `any_reachable`, `internal_reachable`, `reachable_endpoints`, `endpoint_count`; misleading LAN/WiFi guidance removed.
+- **DDL hygiene**: `space_key` column added to `CREATE TABLE published_pages` DDL for self-documenting schema.
+Phase 161 preserves the Phase 160 dispatch model and practical concurrency optimization, and adds:
+- **Manual verification isolation**: real-service verification stays under explicit `tests/verification/` opt-in surfaces.
+- **PDF test-surface recovery**: default collection no longer probes Docker, Redis, or internet at collection time.
+- **Runtime hardening**: package-path timeout fallback, non-targeted row skip for single-row shards, deferred upload retry, unevaluable-row pre-filter, and 1200s single-row shard timeout default.
+- **Confluence SDK call minimization**: `space_key` persistence and session caching, verified-parent dedup, and retry compounding capped per logical operation.
+Phase 160 still delivers practical concurrency optimization on the Phase 158 dispatch substrate:
+- **Checklist pre-cache**: Parent serializes checklist to disk; children read via `VDS_AUDIT_CANONICAL_CHECKLIST_JSON` env var (0 child Confluence calls).
+- **Single-row shards**: `--rows-per-shard 1` dispatches 1 row per job for true row-level backfill via existing `FIRST_COMPLETED + _fill_capacity` loop.
+- **Per-profile timeout**: `~/.vds/config/profile-timeouts.yaml` + `VDS_AUDIT_PROFILE_TIMEOUT_{PROFILE}` env var overrides.
+- **Submission stagger**: `--submission-stagger-ms 100` prevents thundering herd on multi-slot fills.
+- **Risk mitigations**: Per-row failure counter (R9, max 3 profiles before skip), dynamic resume limit (R7, scales with job count).
+Phase 159 RowQueue rewrite deferred — existing dispatch loop is architecturally sufficient with Phase 160 changes.
+

package/skills/vds-skill/runtime/docs/phases/PHASE_125_COMPLETION_AND_MERGE.md

@@ -0,0 +1,212 @@
+# Phase 125 Completion and Merge Report
+
+**Date**: March 21, 2026
+**Status**: ✅ COMPLETE AND MERGED
+**Spec Version**: v4.80.3 (Phase 127 Active)
+**Implementation Phase**: Phase 125 (Skill Integration)
+
+---
+
+## Executive Summary
+
+Phase 125 (Skill Integration Finalization) has been **successfully completed and merged into the master branch** of the VDS Scripts workspace.
+
+### What Was Accomplished
+
+1. **Skill Integration Layer**: Implemented comprehensive skill integration for audit orchestrator
+   - Skill readiness diagnostics
+   - Deterministic recommendation logic
+   - Companion context service
+   - Telemetry separation
+   - CLI integration
+
+2. **Skills Integrated**:
+   - `research-skill` (PRIMARY) - Source-backed research lane
+   - `repomix-skill` (COMPANION) - Bounded companion-context generator
+   - `hexagonal-compliance-skill` (INCLUDED) - Architecture compliance
+   - `llm-analysis-skill` (DOCUMENTED) - LLM-based evaluation
+   - `code-review-graph-skill` (OPTIONAL) - Structural analysis
+
+3. **Verification**: 8/8 checks passing (100%)
+   - All implementation verified
+   - All tests passing
+   - Documentation complete
+
+### Merge Details
+
+- **Source Branch**: `phase125-skill-integration`
+- **Target Branch**: `master`
+- **Merge Commit**: `ca080d0`
+- **Files Added**: 19 files, 5,072 lines
+- **Merge Strategy**: `--no-ff` (non-fast-forward)
+
+### Merge Process
+
+1. ✅ Stashed Phase 126 WIP changes
+2. ✅ Merged Phase 125 branch into master
+3. ✅ Restored Phase 126 WIP changes
+4. ✅ Verified Phase 125 files present
+5. ✅ Created merge documentation
+
+### Commits Merged
+
+```
+ca080d0 Merge branch 'phase125-skill-integration' - Phase 125 Skill Integration Finalization Complete
+17a2b21 docs(phase125): Update completion summary to reflect 100% verification passing
+6088feb fix(phase125): Address GRAPH-READINESS-001 verification issue
+2177001 feat(phase125): Complete Phase 125 Skill Integration Finalization
+f25ace3 feat(phase125): TSK-125.3 - Repomix-skill bounded integration
+```
+
+### Files Delivered
+
+#### Core Implementation (3,500+ lines)
+
+- `skill_recommendation.py` (847 lines) - Deterministic skill recommendation logic
+- `telemetry.py` (367 lines) - Evidence gap vs companion context tracking
+- `companion_context_service.py` (434 lines) - Bounded companion context generation
+- `skill_tools.py` (204 lines) - CLI skill commands
+- `skill_readiness.py` (493 lines) - Skill availability diagnostics
+
+#### Verification & Tests (823 + 232 lines)
+
+- `verify_phase125_skill_integration.py` (823 lines) - Comprehensive verification script
+- `test_companion_context_service.py` (232 lines) - Unit tests
+
+#### Documentation (745 lines)
+
+- `docs/phases/phase125/IMPLEMENTATION_REPORT.md` - Implementation details
+- `docs/phases/phase125/TSK-125.10-11-implementation-summary.md` - Task completion
+- `docs/phases/phase125/profile-patch-ollama-local-anthropic.md` - Profile configuration
+- `docs/phases/phase125_completion_summary.md` - Completion summary
+- `docs/phases/phase125_llm_analysis_skill.md` - LLM skill integration
+
+#### Additional Files
+
+- `PACKAGE_P125B_IMPLEMENTATION_SUMMARY.md` - Package-level summary
+- `analyze_profiles.py` - Profile analysis tool
+
+### Verification Results
+
+**All 8/8 checks passing (100%)**:
+
+```
+✓ [SKILL-READINESS-001] Skill readiness diagnostics available and structured
+✓ [SKILL-RECOMMEND-001] Deterministic skill recommendation logic works
+✓ [ESCALATION-001] Escalation triggers only for weak-grounding rows
+✓ [COMPLEMENT-001] Skill context complements authoritative evidence
+✓ [DECISIVE-001] Decisive evidence skips companion expansion
+✓ [TELEMETRY-001] Telemetry distinguishes evidence gaps from companion context
+✓ [CLI-001] CLI skill commands available
+✓ [GRAPH-READINESS-001] Graph skill readiness logic correct
+```
+
+### Integration Status
+
+| Skill | Status | Role | Integration |
+|-------|--------|------|-------------|
+| `research-skill` | ✅ PRIMARY | Source-backed research | Ready for use |
+| `repomix-skill` | ✅ COMPANION | Bounded context | Ready for use |
+| `hexagonal-compliance-skill` | ✅ INCLUDED | Architecture compliance | Ready for use |
+| `llm-analysis-skill` | ✅ DOCUMENTED | LLM evaluation | Via `--enable-llm` |
+| `code-review-graph-skill` | ✅ OPTIONAL | Structural analysis | Ready for use |
+| `grepai-skill` | ❌ EXCLUDED | Semantic search | Requires Qdrant+Ollama |
+
+### Profile Configuration Notes
+
+Skills toolset is currently **disabled** in production profiles:
+- `omniroute-glm-openai`
+- `omniroute-gpt-openai`
+
+To enable skills toolset:
+```toml
+VDS_AUDIT_LLM__SKILLS_TOOLSET_ENABLED: "true"
+skill_policy.enabled: true
+```
+
+---
+
+## Post-Merge Status
+
+### ✅ Completed Actions
+
+1. **Phase 125 Implementation**: All 11 tasks complete
+2. **Verification**: All 8 checks passing
+3. **Merge**: Successfully merged into master
+4. **Documentation**: Complete implementation and merge documentation
+5. **Phase 126 WIP**: Preserved and restored
+
+### Current State
+
+- **Main Worktree**: `WHO-project/vds-scripts`
+- **Branch**: `master`
+- **Status**: Phase 125 merged, Phase 126 WIP in progress
+- **Clean Working Tree**: No (Phase 126 WIP changes present)
+
+### Next Steps
+
+1. **Phase 126 Development**:
+   - Material completeness service
+   - Remote signal probes
+   - Cache optimization
+
+2. **Optional Cleanups**:
+   - Remove Phase 125 worktree (if no longer needed)
+   - Enable skills toolset in profiles
+   - Update AGENTS.md with Phase 126 status
+
+3. **Documentation Updates**:
+   - Create Phase 126 spec
+   - Update AGENTS.md for Phase 126
+   - Document Phase 126 features
+
+---
+
+## Metrics Summary
+
+- **Total Implementation Time**: ~29 hours
+- **Total Files Added**: 19 files
+- **Total Lines Added**: 5,072 lines
+- **Verification**: 8/8 checks passing (100%)
+- **Test Coverage**: 2 test files (232 lines)
+- **Documentation**: 5 docs (745 lines)
+
+---
+
+## References
+
+### Documentation
+
+- Implementation Report: `docs/phases/phase125/IMPLEMENTATION_REPORT.md`
+- Completion Summary: `docs/phases/phase125_completion_summary.md`
+- Merge Documentation: `docs/phases/phase125_merge_complete.md`
+- LLM Analysis: `docs/phases/phase125_llm_analysis_skill.md`
+
+### Code
+
+- Verification Script: `audit_orchestrator/scripts/verify_phase125_skill_integration.py`
+- Skill Recommendation: `audit_orchestrator/src/vds_audit_orchestrator/engine/skill_recommendation.py`
+- Telemetry: `audit_orchestrator/src/vds_audit_orchestrator/engine/telemetry.py`
+- Companion Context: `audit_orchestrator/src/vds_audit_orchestrator/engine/companion_context_service.py`
+- Skill Readiness: `audit_orchestrator/src/vds_audit_orchestrator/state/skill_readiness.py`
+
+### Tests
+
+- Companion Context Tests: `audit_orchestrator/tests/test_companion_context_service.py`
+
+---
+
+## Conclusion
+
+**Phase 125 (Skill Integration Finalization) is COMPLETE and MERGED into master.**
+
+All implementation tasks are complete, all verification checks are passing, and the code has been successfully merged into the main worktree. The Phase 126 WIP changes have been preserved and restored, allowing development to continue seamlessly.
+
+**Status**: ✅ PHASE 125 COMPLETE AND MERGED
+
+---
+
+*Last Updated: March 21, 2026*
+*Merge Commit: ca080d0*
+*Implementation Phase: Phase 125*
+*Spec Version: v4.80.3 (Phase 127 Active)*